The introduction of biologic agents more than 15 years ago marked the start of enormous progress in therapies for management of psoriasis. Adalimumab and ustekinumab were the first biologic agents administered subcutaneously that surpassed a satisfactory efficacy threshold difference in PASI 75 response rate with respect to placebo greater than 50% (corresponding to a number of patients needed to treat [NNT] < 2).1 Although one of the randomized clinical trials (RCTs) of adalimumab included an active treatment arm with methotrexate (and subsequently, a comparative study of infliximab with methotrexate was published), the first RCT comparing 2 biologic agents (ustekinumab versus etanercept) was published in 2010.2

In 2015, secukinumab was granted marketing authorization; this was the first of a new class of antibodies targeting interleukin 17 (IL-17A). The basic characteristic of anti-IL17A agents is their rapid onset of therapeutic action and having made PASI 90 response the standard benchmark of efficacy, with a difference in response rates compared with placebo in excess of 70%.3 The clinical development programs of these agents have included RCTs with etanercept, adalimumab, and ustekinumab, in addition to placebo, and recently, guselkumab was included as an active comparator in a couple of RCTs.

Guselkumab, approved by the European Medicines Agency (EMA) at the end of 2017, is the first representative of a class of biologic agents that specifically block the p19 subunit of IL-23; other agents in this class include tildrakizumab and risankizumab. The corresponding comparative studies also included etanercept, adalimumab, ustekinumab, and secukinumab, depending on which agent was the most relevant comparator at the time when the trials were designed.

Dermatologists face a difficult choice with a wide range of options, and pivotal RCTs supporting the authorizing agencies' approval do not enable an assessment of relative efficacy (or safety) compared with other therapeutic agents. Other factors may influence outcomes to a greater or lesser extent (depending on the drug), such as previous failure of a biologic agent, body weight, and comorbidities, particularly psoriatic arthritis. On the other hand, neither the populations nor baseline situation (with washout period and PASI scores above 12) used in RCTs adequately represent clinical reality. In any case, we need a means to compare efficacy data from different drugs, given that a cost-effectiveness analysis, taking into account pricing at a given time, is an essential component of any therapeutic algorithm.

First, a systematic review should be conducted according to a well-established methodology,4 to identify published RCTs of the disease in question (in our case, plaque psoriasis). When several RCTs of the same drug are available, the results should be combined using methods that maintain the randomization of each RCT; for these purposes the meta-analysis (MA) methodology was developed. MAs can provide more precise results, but they can also generate errors if differences in design between studies and publication bias are not taken into account.

In the case of psoriasis, the methodology of the studies and the selection criteria in general are very uniform, and this is an advantage although the possibility of study heterogeneity should always be considered. In many cases, the data/outcomes of interest are dichotomous (for example, PASI 90 response: yes/no) and the discussion here will focus on such data, although there are MA methods that can be applied to continuous data/outcomes, ordinal outcomes or combinations thereof. Finally, sensitivity analyses should be employed (changing the conditions determining which RCTs to include) to confirm that the findings are robust and can thus be used to guide therapeutic or reimbursement decisions.

With a pairwise MA, data from several RCTs of drug A compared with placebo (or drug B) can be combined; on combining different studies, it may be possible to detect a significant difference that would otherwise not be apparent because the individual RCTs are underpowered.

In MAs, a statistical parameter representative of the intervention effect (treatment) is calculated for each of the studies; for dichotomic variables, the odds ratio (OR), the risk ratio (RR), or the risk difference (probability or percentage of response) between the drug and its comparator (usually placebo) are used. For example, if a drug can achieve a PASI 90 response in 633 of 1000 patients (63.3%), compared with 25 of 950 (2.6%) for placebo, the OR is (633×925)/(25×367)=63.8; the RR is (633×950)/(25×1000)=36.1; the risk difference (63.3-2.6)=60.7% (or 0.67), and 3 patients need to be treated to achieve a PASI 90 response in 2 (NNT=1/0.67=1.5). OR and RR are relative measures, whereas the risk difference and NNT are absolute measures; in general, the relative measures are more consistent among studies and do not have numerical limits (a probability cannot be negative). In general, clinicians tend to overestimate the treatment effect when expressed as an OR, although the differences between OR and RR are small for uncommon events (with a probability of approximately < 20%). This is not the case for treatment response in RCTs of psoriasis.

Essentially, there are 2 types of statistical model for MAs, depending on whether each RCT is considered to measure an unchanging drug effect or whether the effects in each RCT follow a random distribution.5

• •

  The fixed-effects model assumes that there is no variation in the relative treatment effect between different RCTs, and that the differences observed are due only to chance. In this case the aim is to calculate the true treatment effect. In general, the Mantel-Haenszel statistical method is used, particularly for small sample sizes, but the inverse variance method and the Peto method can also be used. These methods are more powerful and less biased when there are studies without events (for example, mortality), the intervention effect is small (probability <1%), and the size of the treatment arms is balanced.

• •

  The random-effects method assumes that the treatment effect can vary from one RCT to another (the outcome of each RCT would follow a normal distribution) and the combined effect would be the mean of a normal distribution whose standard deviation would reflect heterogeneity. In the case of the random-effects model, the DerSimonian-Laird method is generally used.

The effect measure for any of these methods can be OR, RR, or risk difference, except when the Peto method is used, in which case only ORs can be combined. The effect of each RCT is represented in an effects diagram or forest plot, by means of blocks with horizontal lines that indicate the confidence interval (CI), generally at a level of 95%. The size of each block often represents the weight assigned to each study in the MA. A common way to weigh RCTs is according to the reciprocal of the variance or square of the standard error: large studies are given more weight because their standard error is smaller. The summary effect, or weighted mean of the effects of each intervention, is represented by a diamond (with a height proportional to the number of subjects enrolled in the RCT) (Fig. 1).6

Fig. 1.

Example of forest plots corresponding to 2 pairwise random-effects MAs; the effects (RR for PASI 90 response) were estimated using the Mantel-Haenszel ((M-H)) method and different measures of heterogeneity have been calculated.

Modified from Cameron et al.6

(0.23MB).

There are different measures of heterogenicity (due to differences in the characteristics of the populations in each study [clinical diversity], study design, outcome measures of interest, and bias [methodological diversity]) between the different RCTs included in an MA; for example, the Cochran Q test (χ2) with its corresponding P value, or I2 with its corresponding CI, which is used in Cochrane reviews. I2 is calculated from the Cochrane Q value as 100% × (Q – degrees of freedom)/Q, and it describes the percentage variability in the effect estimates due to heterogeneity, plus the random sampling error. Negative values are considered as 0% and indicate that heterogeneity is not observed; values below 25% indicate that the RCTs have low heterogeneity; values above 50% indicate high heterogeneity. Another measure of heterogeneity that is used for random-effects MAs is τ2.

The heterogeneity among different RCTs can be due to treatment interactions with different covariates (baseline values of weight, age, comorbidities, placebo effect, etc) that determine bias.7 Covariates can also be interdependent; for example, weight may modify the response not only of treatment but also of placebo.8 There are statistical measures of the extent of bias and metaregression models that take into account these interactions, although it is preferable to use indirect comparisons with individual data (see below).

Pairwise MAs provide a pooled estimate of a treatment effect compared with placebo, and the 95% CI indicates that the real or average value of the effect lies within the interval with 95% probability, but it does not enable rigorous comparison of the relative effect of 2 treatments, particularly when CIs overlap.9

The relative efficacy of different treatments or therapeutic interventions can be assessed with comparative RCTs (direct comparisons), MAs with multiple RCTs (indirect comparisons), and patient registry data.

Head-to-head (H2H) comparative RCTs require random assignment to treatment and double-blind assessment of efficacy, but even if the inclusion criteria are identical and the treatment arms are largely balanced, biases may still be present. For example, even assuming that there are no meaningful changes in the potency of the molecule over time, the absence of a placebo arm can increase therapeutic response compared with pivotal RCTs, as observed in some equivalence trials.10 The choice and timing of the endpoint assessment to determine the outcome of an RCT will often favor the sponsor’s desired outcome.11,12 When there are differences between cutaneous response and articular response between 2 drugs, the choice of a composite outcome measure can also favor one of the drugs.13

Even when H2H RCTs are available, both clinicians and drug evaluation agencies are placing increasing importance on the different methods of indirect comparison between treatments, particularly when multiple options are available. By combining direct and indirect comparisons, we can obtain a more refined estimate of the relative value of each drug, provided these comparisons meet certain conditions discussed later.14

The different methods available for indirect comparisons of treatments include15:

• •

  Adjusted or anchored indirect treatment comparisons (ITCs), comparing aggregate data from RCTs of drug A with aggregate data from the corresponding placebo arm or another drug B which has been compared with placebo.

• •

  Network MA (NMAs), comparing aggregate data from different drugs with placebo (and between them if H2H RCTs are available), with or without adjustment for baseline characteristics and in particular different placebo responses.

• •

  Indirect comparisons with individual data on a treatment adjusted for the aggregate characteristics of others (matching-adjusted indirect comparisons, or MAIC), weighting individual patient data such that they become equivalent to RCTs for which only aggregate data are available.16–19 There are other methods for indirect comparison with data from individual patients, and recommendations have been published with a view to their use both in approval processes and reimbursement evaluations.20

• •

  Indirect comparisons with individual data for each treatment arm (individual patient-level data, or IPD).

Randomization in RCTs enables the treatment effect to be separated from other factors and ensures baseline risk is balanced across treatment arms, even in RCTs performed over different periods. To maintain the effect of randomization in multiple comparisons, Bucher et al.21 introduced the ITC method, using OR as the effect measure, although this is only applicable to indirect comparisons of treatments that have been compared with another common comparator.15 Subsequently, Lumley et al.22 developed a technique (NMA) to compare 2 treatments via more than 1 common comparator, including a measure of the possible inconsistency or discrepancy in results; the approach was subsequently further developed.23,24 Finally, Lu and Ades25 incorporated direct comparisons (H2H RCTs) in a more sophisticated method denoted mixed treatment comparisons (MTCs), which enables different treatments to be ranked according to the probability that they are better or worse.

Indirect treatment comparisons require the clinical RCTs to be linked to form a network, whether with open or closed loops, if they include H2H RCTs (direct evidence, as well as indirect evidence) (Fig. 2). When we have H2H RCTs for some treatments, or even RCTs with common treatment arms in addition to placebo, NMAs are used to obtain estimates of the relative effect of different treatments with greater precision than pairwise MA and rank them coherently, taking into account possible inconsistencies between the results of direct and indirect comparisons.

Fig. 2.

Classification and nomenclature of indirect comparisons of interventions (treatments) and their diagrams. In the broadest sense, all indirect comparisons are NMAs, unlike traditional MA, which do not simultaneously compare several treatments but rather pool different RCTs comparing 2 interventions, for example, treatment versus placebo. Bucher21 introduced ITCs to indirectly compare 2 treatments that have been compared directly with placebo in different RCTs. Lumley et al.22 developed NMAs in the strict sense of the term to compare 2 treatments via more than 1 common comparator. Lu and Ades25 introduced the method of MTCs, which include both indirect comparisons and direct comparisons by means of H2H RCTs (closed loops) and enable a probable efficacy ranking of different treatments to be established.

(0.15MB).

There are different guidelines for publishing RCTs, MAs, and NMAs; PRISMA is the most relevant in the latter case.26 In NMAs, it is essential to provide information on the network structure, and to help with this task, there are specific programs that produce diagrams in which the size of the circles is proportional to the number of patients randomized to each treatment (or placebo) and the width of the lines is proportional to the number of studies included in each comparison (Fig. 3).27

Fig. 3.

Example of an NMA network diagram for PASI 90 results at 12 or 16 weeks of biologic treatment for psoriasis.27 The size of the nodes represents the sample size of the interventions and the widths of the lines the number of studies included.

Abbreviations: ADA, adalimumab; APR, apremilast; BIW, twice weekly, BRO, brodalumab; ENT, etanercept; GUS, guselkumab; IFX, infliximab; IXE, ixekizumab; PBO, placebo; QW, once a week; Q2W, every 2 weeks; SEC, secukinumab; UST, ustekinumab.

Modified from Cameron et al.6

(0.29MB).

NMAs can be performed within a frequentist or Bayesian framework.28

In the former, the result of the NMA is expressed as a point estimate of the relative efficacy of each treatment compared with the others, with a 95% CI; if the NMAs were repeated, the real value of the effect would lie within the CI 95% of the time; this does not mean that the probability of the real value lying within the CI is 95%.

The Bayesian approach implies a subsequent distribution of probabilities based on the results of NMAs, which enables the different treatments to be ranked according to their relative efficacy,29 thereby facilitating pharmacoeconomic decision making. When treatment effects are evaluated, it is generally assumed that the probability a priori is not informative, although previous NMA data can be incorporated.30 Bayesian NMAs require extensive calculations with statistical packages that can run Markov Chain Monte Carlo (MCMC) simulations. When significant heterogeneity is present among the RCTs included in the NMA, it is recommended to adjust for baseline risk (for example, placebo effect) taking into account multiple covariates, whether observed or not, and these are used to optimize the model.6 The equivalents of CIs for the relative effects of the different pairwise comparisons in the league tables are denoted by credibility intervals (CrIs) in the Bayesian framework.

The results of NMAs allow comparison of effects (efficacy for example) for each treatment relative to another. The results are generally presented in the form of league tables (Fig. 4).31

Fig. 4.

Example of league table for a frequentist random-effects NMA showing RR outcomes for PASI 90 response adjusted for baseline risk. The treatments are ordered left to right by decreasing SUCRA rank. The effect estimate (efficacy) is the RR of the treatment defined by the column with respect to that defined by the row; if the 95% CI does not include 1, RR>1 favors the treatment defined in the column, RR<1 favors treatment defined in the row.

Extracted from Cameron et al.6

(0.35MB).

The ranking of the different treatments is based on calculation of surface under the cumulative ranking curve (SUCRA) (Fig. 5),27 which is expressed as a percentage and can be interpreted as the probability that all the other treatments are inferior.32 The estimated ranking for each drug in the NMA can also be represented with their corresponding 95% CrI in the so-called ranking plots. Although the SUCRA approach was developed in a Bayesian framework, some programs allow SUCRA calculations in frequentist NMAs; recently, an index has been proposed (P-score)33 which allows a similar ranking to be established in the frequentist framework.

Fig. 5.

Illustration of SUCRA values.27 The ranking plot (not represented) would give the probability of the rank order, that is, that each treatment was the best, second, third, etc., following a normal distribution or another type of distribution. SUCRA is the surface under the cumulative ranking curve. The horizontal axis represents the possible rank for each treatment (from being the best [1] to the worst [14 in this case] from left to right). On the vertical axis, the cumulative probability that each treatment is the best option, the first or second best option, one of the best 3 options, etc. If a treatment has 100% probability of being the best, the y value will be 1 (100%) and the curve will be a horizontal line (100% SUCRA); the curve for a treatment (for example, placebo) that had 100% probability of being the worst would be represented by a horizontal line to the last position where it would increase to 1 (SUCRA 0%). The intermediate SUCRA values can be considered as the percentage of area under curve compared to the rectangle that represents 100%, and enable different treatments to be ranked. Another way of interpreting SUCRA for a given treatment is as the probability (expressed as a percentage) that the other treatments are worse.

A, brodalumab 140mg; B, brodalumab 210mg; C, guselkumab 100mg; D, guselkumab 50mg; E, ixekizumab 80mg Q2W; F, ixekizumab 80mg Q4W; G, placebo; H, risankizumab 150mg; I, secukinumab 150mg; J, secukinumab 300mg; K, tildrakizumab 100mg; L, tildrakizumab 200mg; M, ustekinumab 45mg; and N, ustekinumab 90mg.

(0.34MB).

Both ITCs and NMAs use aggregate data, as provided in the publications, in a systematic fashion, following the guidelines of CONSORT and SPIRIT.26 When patient-level data for one of the treatments and placebo in several RCTs are available, an indirect MAIC-type comparison can be performed, and when individual data are available for several treatment arms, an IPD analysis can be performed.

In another publication, an indirect comparison of guselkumab was performed with ustekinumab using IPD from different RCTs with the same sponsor and adjusting for differences in covariates using a multivariate regression model.45 This publication also illustrates the potential limitations of the inclusion of RCTs with arms in which there is a partial unblinding or enrichment of a given arm because of switching following prior response, but the sensitivity analyses supported the results obtained.