The randomized controlled trial (RCT) is the most rigorous type of methodological design and the best way of determining whether a cause-effect relation exists between an intervention and the result or outcome being assessed. RCTs also provide the raw material for systematic reviews and meta-analyses. However, the value of such studies depends on the quality and methodological rigor of their design and implementation.

In recent decades, the field of dermatology has seen a substantial increase in experimental clinical research. However, this upturn in the volume of research has not been accompanied by a corresponding improvement in trial design and methodology. Several studies have reported that the RCTs published in the dermatology literature tend to fall below acceptable standards.1–4

The Consolidated Standards of Reporting Trials (CONSORT) statement was first published in 1996 to improve the quality of reporting of clinical trials worldwide.5 The CONSORT statement includes a checklist designed to improve the reporting of RCTs, which also, indirectly, throws light on the study's quality and scientific rigor.

An improvement in the scientific quality and reporting of RCTs might have been expected following the implementation of CONSORT and the publication of the Medical Research Council Guidelines for Good Clinical Practice in Clinical Trials (available from: http://www.fda.gov/downloads/Drugs/Guidances/ucm073122.pdf and http://www.mrc.ac.uk/documents/pdf/good-clinical-practice-in-clinical-trials/). However, the evidence reveals the continued presence after 1997 of serious flaws in the design and reporting of clinical trials.6 The problem has also been observed in trials in the Spanish-language dermatology literature published after 1997. A study carried out in Spain found that only 6 (25%) of the 24 clinical trials found in the dermatology journal with the highest impact in that country—Actas Dermo-Sifiliográficas—were classified as being of high quality.3

In this context, the aim of the present study was to assess the methodological quality of the experimental clinical research in dermatology published in Spanish to facilitate an analysis of the strengths of these studies and the challenges that must be overcome. We analyzed the RCTs identified by a recent study that handsearched Spanish-language dermatology journals.7

The present study complements that work by analyzing the methodological quality of the RCTs published between 1997 and 2012 using the appropriate Cochrane Collaboration tools and a review of the reporting of conflicts of interest and funding sources.

To assess the methodological quality of the RCTs published in Spanish-language dermatology journals between 1997 and 2012.

Of the 28 journals that fulfilled the criteria for eligibility, 21 were eventually included in the study: 5 from Spain and 16 from Latin American countries. Of these 21 journals only Actas Dermo-Sifiliográficas is currently indexed on both MEDLINE and EMBASE. Four others are indexed on EMBASE: Dermatología Revista Mexicana, Argentina de Dermatología, Medicina Cutánea Ibero Latinoamericana and Piel.7

The total number of journals included and excluded, and the reasons for the choices made have been described in an earlier article (Fig. 1).7

Figure 1.

Flow chart showing the process used to select dermatology journals according to inclusion and exclusion criteria. Source: Sanclemente G, Pardo H, Sánchez S, Bonfill X. Identificación de ensayos clínicos en revistas dermatológicas publicadas en español. Actas Dermosifiliogr. 2015;106:415-422).

(0.62MB).

Identification of Clinical Trials

Seventy RCTs published between 1997 and 2012 were identified in the 21 journals studied: 73% (51) in the 16 Latin American journals and 27% (19) in the 5 Spanish journals (Table 1) (Appendix 1). The Latin American journals that published the largest number of RCTs were Dermatología Revista Mexicana (16), Dermatología Peruana (9), and Revista Chilena de Dermatología (5) (Table 1). The Spanish journals that published the most RCTs were Actas Dermo-Sifiliográficas and Piel, with 8 each (Table 1).

Table 1.

Randomized Controlled Trials (RCTs) Identified in Spanish and Latin American Dermatology Journals.

	Journal Name	Periods Not Assessed Because No Copies (Print or Electronic) Available	Number of RCTs Identified
1	Dermatología Revista Mexicana	–	16
2	Dermatología Peruana	–	9
3	Actas Dermo-Sifiliográficas	–	8
4	Piel	–	8
5	Revista Chilena de Dermatología	–	5
6	Dermatología Cosmética, Médica Y Quirúrgica	–	4
7	Revista del Centro Dermatológico Pascua	–	4
8	Revista Asociación Colombiana de Dermatología	–	4
9	Medicina Cutánea Ibero-Latino-Americana	–	3
10	Folia Dermatológica Peruana	–	2
11	Dermatología Argentina	–	2
12	Folia Dermatológica Cubana	–	2
13	Dermatología Venezolana	–	2
14	Revista Argentina de Dermatología	–	1
15	Archivos Argentinos de Dermatología	–	0
16	Actas de Dermatología y Dermatopatología	–	0
17	Revista Dominicana de Dermatología	1997-2009	0
18	Revista Sociedad Ecuatoriana de Dermatología	1997-2002, 2005, 2008, 2009, 2011, 2012	0
19	Gaceta Dermatológica Ecuatoriana	1999-2012	0
20	Actualidad Dermatológica	–	0
21	Revista Fontilles	1997-2002	0
Total			70

Most of the trials reviewed were classified as having a high risk of bias because the authors failed to report the information needed to assess the quality and methodological rigor of the trial (Table 2). A small percentage of trials had a low risk of bias in the domains assessed (Table 2) (Fig. 2).

Table 2.

Assessment of Methodological Aspects of Randomized Controlled Trials (RCTs) Published in Spanish and Latin American Dermatology Journals.

Methodological Aspect	No. (%)	95% CI
Allocation concealment
Unclear/Not reported	48 (68.6)	57.73%-79.47%
Yes	14 (20)	10.63%-29.37%
No	8 (11.4)	3.95%-18.85%
Sequence generation
Unclear/Not reported	62 (88.6)	81.15%-96.05%
Yes	6 (8.6)	2.03%-15.17%
No	2 (2.8)	–1.06%-6.66%
Blinding of participants/personnel
Unclear/Not reported	44 (62.8)	51.48%-74.12%
Yes	11 (15.7)	7.18%-24.22%
No	15 (21.5)	11.88%-31.12%
Blinding of outcome assessors
Unclear/Not reported	44 (62.8)	51.48%-74.12%
Yes	11 (15.7)	7.18%-24.22%
No	15 (21.5)	11.88%-31.12%
Incomplete outcome data
Unclear/Not reported	42 (60)	48.52%-71.48%
Yes	13 (18.5)	9.4%-27.6%
No	15 (21.5)	11.88%-31.12%
Selective outcome reporting
Unclear/Not reported	12 (17.2)	8.36%-26.04%
No	44 (62.8)	51.48%-74.12%
Yes	14 (20)	10.63%-29.37%
Other sources of bias
Unclear/Not reported	35 (50)	38.29%-61.71%
No	5 (7, 15, 8)	7.26%-24.34%
Yes	30 (42.9)	31.31%-54.49%
Funding sources
Not reported	55 (78.6)	68.99%-88.21%
Reported	15 (21.4)	11.79%-31.01%
Conflicts of Interest
Not reported	65 (92.8)	86.74%-98.86%
Reported	5 (7.2)	1.14%-13.26%

Figure 2.

Risk of bias of the RCTs analyzed (graphic designed using Review Manager, version 5.2, Copenhagen, The Nordic Cochrane Centre, The Cochrane Collaboration, 2012).

(0.21MB).

The authors of 15 RCTs reported sources of funding and only 2 did so in the required manner (Ramirez-Bosca et al. and Pinto et al.) (Appendix 1). The authors of 5 studies reported conflicts of interest, but only 1 of these reports conformed to accepted standards (Ramirez-Bosca et al.) (Appendix 1). Of all the journals assessed, only Actas Dermo-Sifiliográficas,since 2008, requires authors to report trials in accordance with the CONSORT guidelines; however, none of the authors reported in the body of the article whether or not the CONSORT checklist had been used to guide the reporting of the study.

Figure 2 is a summary of the risk of bias for the RCTs identified.

The objective of this study was to assess the methodological quality of the RCTs published in Spanish-language dermatology journals between 1997 and 2012. Our findings show that the risk of bias was high in the clinical trials published in the Spanish-language dermatology literature in that period, primarily because authors failed to report on important methodological aspects of their work. Although this shortcoming had already been described in earlier studies focusing on specific dermatology journals in Spanish,9 this is the first comprehensive analysis that covers all the dermatology journals publishing RCTs in Spain and Latin America. Our findings are similar to those of authors who studied RCTs in the English-language dermatology literature or RCTs on diseases such as perioral dermatitis and atopic dermatitis.4,10,11

The presence of such flaws in RCTs is of particular concern because this type of study is considered to be a gold standard for the assessment of the efficacy and safety of an intervention. Consequently, the implication is that dermatological practice today (at least that predicated on evidence from the studies assessed) may be based on information gathered in a non-systematic manner or on clinical experiments lacking control groups.12 We also detected a mismatch between the outcomes typically assessed and those that might interest patients. For example, many dermatology studies now incorporate variables relating to quality-of life because of the considerable interest of patients in this outcome in relation to dermatological treatments.13–15 However, it is striking that quality-of-life was assessed in only 1 of the 70 RCTs identified.

The methodological aspects least often reported were random sequence generation and allocation concealment; authors also failed to report on sources of funding and possible conflicts of interest. Our findings, which are similar to those observed by other authors in journals that endorse CONSORT reporting as well as in those that do not,6,16,17 highlight shortcomings in the scientific rigor with which the RCTs were designed and reported. In the future, experimental clinical research published in Spain and Latin America in the field of dermatology needs to be considerably improved both in the design and the reporting of results (endorsement and application of the CONSORT guidelines).

The starting point for an unbiased study is the use of a mechanism that ensures that all the patients have the same probability of belonging to one group or the other, and that adequate concealment of the allocation sequence prevents selective recruitment of patients according to prognostic factors (guidelines available from http://handbook.cochrane.org/). In fact, it has been shown that inadequate random sequence generation in an RCT can result in an overestimation of the effect of the treatment of up to 12%,18 while inadequate allocation concealment may increase the effect up to 18%. Furthermore, the fact that a clinical experiment is classified as randomized does not, in and of itself, guarantee that the study fulfils the methodological standards associated with this type of study.19

The only journal included in this study that requires authors to comply with the CONSORT statement when reporting clinical trials is Actas Dermo-Sifiliográficas. This endorsement may explain the higher methodological quality of the RCTs published recently by that journal. However, it has been observed that, despite improvements in reporting of RCTs when this tool is used, the completeness of reporting of trials continues to be suboptimal in terms of ensuring a better quality of study.6

Of note is the fact that almost none of the trials identified provided any information on sources of funding or conflicts of interest. Complete reporting of both of these aspects is essential since the results of the trial may be affected by the personal interests of the researcher or the funder of the study (very often a pharmaceutical company).20–22 Transparency is important because it is common in the dermatology literature to find selective reporting of endpoints, a practice which in most cases leads to the overestimation of positive outcomes.23 This practice may be associated with the presence of conflicts of interest. Therefore, in the future careful assessment of these characteristics will be essential in the studies published in Spanish-language dermatology journals.24

One of the principal strengths of the present study was that 21 dermatology journals published in Spanish were handsearched to identify RCTs. The clinical trials identified will shortly be included in the Cochrane Central Register of Controlled Trials (CENTRAL), making them available for future systematic reviews and other summary documents. As reported by the earlier article, which identified the RCTs7 analyzed in the present study, finding 70 RCTs and retrieving the full texts of those articles would have been impossible through an electronic search because only 1 journal is indexed on MEDLINE (Actas Dermo-Sifiliográficas) and only 4 are indexed on EMBASE7 (Dermatología Revista Mexicana, Revista Argentina de Dermatología, Medicina Cutánea Ibero Latinoamericana and Piel). Another strength of the present study was the duplicate analysis of the quality of the RCTs and the use of internationally-recognized and validated Cochrane tools. (Available from: http://handbook.cochrane.org/chapter_8/8_assessing_risk_of_bias_in_included_studies.htm).

The main limitation of this study was the impossibility of assessing all the volumes and issues of 3 journals: 2 published in Ecuador and 1 published in the Dominican Republic. However, it is unlikely that our results would have differed significantly with a complete analysis of these 3 journals since no RCTs were found in the issues we were able to review. Furthermore, none of those journals have endorsed the CONSORT statement or require its use. Another limitation was the variability of the endpoints and the way these were measured in the RCTs identified. This variability led to a high level of heterogeneity among the studies, making it difficult to quantitatively summarize the results in a meta-analysis

In conclusion, the risk of bias of the clinical trials published in Spanish-language dermatology journals between 1997 and 2012 was high, mainly because the study reports provided insufficient information on which to base any assessment of the quality and methodological rigor of the studies. Moreover, in many cases the authors failed to report on sources of funding and possible conflicts of interest. Complete reporting of all methodological aspects of trials is recommended, as this would allow readers to detect possible sources of bias and design flaws. A complete description of the study is important because it facilitates proper analysis of the evidence and because it ensures that a trial is not classified as having a high risk of bias solely because of omissions in the information provided. Complete reporting will benefit patients—the foundation of evidence-based dermatological practice—and will contribute to more effective decision-taking in this field of practice. Finally, and as a future strategy, we plan to contact the publishers of the dermatology journals analyzed with a view to standardizing prospective tools for the identification of the RCTs published in their journals. The implementation of such a system will facilitate continual updating of this work, thereby obviating the need to repeat the manual search in the future.

Grupo de Investigación Dermatológica (GRID), Facultad de Medicina, Universidad de Antioquia, Medellín, Colombia.

The authors declare that they have no conflicts of interest.