The impact factor (IF) has come to carry increasing weight in journal evaluation and management, and editorial boards logically discuss strategies to improve theirs. One such a strategy is selecting article types that will attract the largest number of citations. As some journals have decided to prioritize certain article types, they have reduced the number of pages allotted to others, such as case reports.1

Initially conceived as an indicator for comparing the influence, or impact, of scientific journals,2 the IF is now seen as a reflection of a journal's quality and prestige, even though that use may be inappropriate.3 The Web of Knowledge of the Institute for Scientific Information (ISI) publishes an annual list of IFs for all journals indexed in their database, the Web of Science. The lists are known as the Journal Citation Reports. To be selected for indexing in the ISI's Web of Science, a journal must meet certain criteria, such as the regular publication of issues, the ease with which cited sources can be recovered, the language of publication (full text in English, sources in English), and content (a range of articles of scientific interest that includes emerging topics).4

A journal's IF is calculated for a calendar year (2015, for example) and is expressed as the quotient of a fraction in which the numerator is the number of citations the journal received that year by articles indexed in a specific database (such as the ISI's) and the denominator is the number of articles published by the journal in the 2 preceding years (ie, those published in 2014 and 2013 for a 2015 IF). Thus, the quotient reflects the average overall citation rate for a journal's articles in a given year.5

The numerator includes all references made to any article published in a journal, making the IF a direct reflection of the number of citations attracted. The more articles the journal publishes, the more articles can be cited, favoring a higher IF. Journals on more specialized clinical topics like dermatology will, a priori, have lower IFs than general medical journals such as the New England Journal of Medicine.6 To illustrate the effect of specialization, it is sufficient to note that the highest IF in dermatology in 2015 (6.915) was earned by the Journal of Investigative Dermatology. In contrast, the New England Journal of Medicine's IF for the same year was 59.558.7

The denominator includes only the articles an index such as the ISI's classifies as citable. Given a high IF's influence on visibility and even on a journal's income, many editorial boards that prioritize this metric will apply various strategies to achieve it. If, for example, a report is published as a letter to the editor, the ISI will include a reference to it in the numerator, but the letter will be omitted from the denominator because it is not classified as a citable publication. Journals therefore publish information in letters to lower the number of citable articles and thereby raise the quotient. Other ways to raise an IF include publishing reviews, original articles from well-known research groups, and studies of common diseases; journals may even decide to reject certain articles they believe will attract few citations.8 For this reason many researchers have explored how article type influences citation rates. For example, a study comparing citations received by ear-nose-throat journals in the UK showed that review articles received 2-fold more citations than case reports on average.9

Some dermatology journals have increased their IFs surprisingly in recent years. Contact Dermatitis's rose from only 2.701 in 2005 to 5.514 in 2015, for example, possibly because they began publishing more articles of general interest in areas like molecular biology, vascular biology, immunology, infections, or conjunctive tissue conditions, among other topics. Such articles are believed to attract more citations. Some journals have also eliminated abstracts from short case reports.10

A limitation of the ISI IF, which is published only by Thomson Reuters, is that it is difficult for outsiders to replicate, obliging other bibliometric services to develop their own metrics. The exact criteria Thomson Reuters applies to identify citable articles for specific journals remain unclear, explaining the difficulty or impossibility of replication. This problem has been discussed in the literature.11,12

Our aim was to describe the influence of different article types (case reports, original articles, reviews, and “other” types) on the IFs of dermatology journals.

We found a total of 6934 articles published in 2013 and 2014 by the 5 journals. After the classification process, 5099 were excluded, leaving 1835.

Over half the articles retrieved for many of the journals were excluded. For example, we set aside 1830 of the 2528 BJD publications retrieved and 526 of the 785 JAMA Dermatol publications. Of the 378 published by Actas Dermosifiliogr, we set aside 170.

The publication of many proceedings issues corresponding to international conferences accounted for the large number of exclusions. In 2013 alone, the 93rd Annual Meeting of the British Association of Dermatologists and the 27th Annual Meeting of the British Society for Paediatric Dermatology, among other conferences, led to the publication of 1239 titles.

Furthermore, many articles were published without abstracts. Most of them were in sections where case reports appeared, including the section for letters to the editor.

Citations and Impact Factor Calculation

Table 1 shows the discrepancy between the number of citations according to ISI reporting and the number found by our search in Scopus.

Table 1.

Excluded and Included Articles and Citation Data.

Journal	Articles Published in 2013	2015 Citations of 2013 Articles	Articles Published in 2014	2015 Citations of 2014 Articles	Total No. of Excluded Articles	Total No. of Included Articles	Total No. of Citations, Scopus)	Total No. of Citations, ISI	Calculated IF, Scopus Data
Actas Dermosifiliogr	81	124	127	96	170	208	220		1.06
BJD	341	1611	357	848	1830	698	2459	3013	3.52
Contact Dermatitis	79	313	77	331	166	156	644	888	4.13
JAAD	262	1008	251	1080	2407	514	2088	2889	4.07
JAMA Dermatol	129	645	130	433	526	259	1078	1320	4.16

ABBREVIATIONS: Actas Dermosifiliogr, Actas Dermo-Sifiliográficas; BJD, British Journal of Dermatology; IF, impact factor; ISI, Institute for Scientific Information; JAAD, Journal of the American Academy of Dermatology; JAMA Dermatol, JAMA Dermatology.

Citations were not normally distributed. Figure 1 shows those received by each type of article in each journal expressed as percentages of the total: a high degree of bias can be seen given that most articles were not cited at all.

Figure 1.

Distribution of citations according to article type for each journal. Actas refers to Actas Dermo-sifiliográficas; BJD, British Journal of Dermatology; Contact, Contact Dermatitis; JAAD, Journal of the American Academy of Dermatology; and JAMA Dermatol, JAMA Dermatology.

(0.31MB).

The highest IF was earned by JAMA Dermatol (4.16), followed by Contact Dermatitis (4.13), JAAD (4.07), and BJD (3.52) (Table 2). The IF of Actas Dermosifiliogr (1.06) was substantially lower.

Table 2.

Differences in Journal Rankings Between ISI-Published IF and Our Scopus-Based IF.

	Journals, Ranked by the ISI IF	ISI-Published IF	Journals, Ranked by the Scopus-Based IF	Our Scopus-Based IF
1	Contact Dermatitis	5.69	JAMA Dermatol	4.16
2	JAAD	5.62	Contact Dermatitis	4.13
3	JAMA Dermatol	5.09	JAAD	4.07
4	BJD	4.32	BJD	3.52
5	–		Actas Dermosifiliogr	1.06

ABBREVIATIONS: Actas Dermosifiliogr, Actas Dermo-Sifiliográficas; BJD, British Journal of Dermatology; IF, impact factor; ISI, Institute for Scientific Information; JAAD, Journal of the American Academy of Dermatology; JAMA Dermatol, JAMA Dermatology.

We found that different types of articles exercise different degrees of influence on a journal's IF. Narrative reviews and original articles tend to raise an IF whereas case reports lower it. This effect was seen in all the journals we analyzed (Table 3). Moreover, while the degree of influence of each article type on overall IFs tended to be similar across the journals, there were small differences in the ratios between article-type IFs and a journal's overall IF. For example, the effect of case reports on lowering the IF was more attenuated in Contact Dermatitis and JAMA Dermatol than in the other journals. Moreover, the original articles published by JAMA Dermatol had the strongest positive influence on its IF, while narrative reviews made the strongest contribution to the IFs of BJD and Actas Dermosifiliogr. The effect of the narrative reviews could not be analyzed for JAMA Dermatol because that journal published only 2 of them in 2013 and 2014. Likewise, the effect on IF of articles in the category “other” could not be discerned because this category contained such diverse types.

One limitation of our study was the difficulty of replicating the IF calculations published by the ISI, an issue that has been discussed elsewhere12 and remains an important problem today, considering the emphasis now placed on this metric. Even the slight differences in IF that emerge when small changes are made in the method of calculation affects a journal's rank. Table 2 shows that the ISI-calculated IFs differ considerably from the ones we calculated based on Scopus. In fact, although Scopus generates its own indicator of influence, the SCImago Journal Rank (SJR), it is also difficult to replicate because it uses citations accumulated over 3 years and is weighted. The 2015 SJRs published for the 5 journals we studied were as follows: Actas Dermosifiliogr, 0.623; BJD, 2.021; Contact Dermatitis, 1.010; JAAD, 2.242; JAMA Dermatol, 1.485.13 The Scopus-based IFs we calculated were lower than the ISI-published figures for 2015. The resulting differences in rank (Table 2), show that JAMA Dermatol is the leading journal according to our calculations, whereas the ISI IF puts that journal in third place, behind both Contact Dermatitis and JAAD. Actas Dermosifiliogr, with a smaller readership, has a lower IF than the others, although it has tended to rise in recent years, from 0.05 in 200714 to 1.06 in 2015.

The lack of methodological certainty when assessing a journal's influence is related to variation in citations retrievable from different indexes. The citations we retrieved from the Scopus database did not coincide with those indexed by ISI. It is also difficult to know exactly which citable articles the ISI includes in their denominators. We made an effort to obtain figures that would be similar to those in the ISI denominators, but the number of citations we found for the numerator was lower.

One noteworthy observation was the large number of articles excluded from IF calculations. Some exclusions were the result of the publication of conference-abstract issues, but we also had to exclude many articles published without abstracts. It is possible that journals omit abstracts intentionally to influence the IF by publishing reports as “notes” or in other sections that are not incorporated into the denominator.

The IF reflects the average citation rate across the journal as a whole, whereas actual citations are not normally distributed and all articles logically do not contribute equally to the IF. The most cited articles normally account for only a small percentage of a journal's publications, and some studies have shown that 50% of citations come from 15% of the articles.3 Including only a few highly cited articles in the calculation increases the IF considerably, since it is an average and therefore highly susceptible to the influence of values at the extremes. It would be more reasonable, therefore, to assess median impact. Furthermore, all citations of an article carry the same weight in IF calculations,15 as no distinction is made between those citing new research and those citing other article types of less importance. Likewise, references to articles being criticized (ones that the scientific community has rejected as valuable) also count toward improving an IF.

Another limitation of our study is that we calculated the IFs for only a single year. Journals change, adopting new strategies to improve their metrics, and the results of studies like ours may therefore vary over time.

Our findings show that top-ranked journals publish fewer case reports. Nabil and Samman,16 who also studied whether high-IF journals really publish fewer case reports, analyzed hypothetical IF calculations in which they discounted both citations received from case reports and the number of case reports published. They concluded that the IFs of journals that published more case reports were indeed negatively affected by that editorial decision.

Our study confirms how difficult it is to replicate IF calculations, revealing a weakness of an indicator that carries so much weight in publishing today. IFs can be manipulated to a certain degree by publishing more articles without abstracts, as such articles still attract citations even though they are excluded from the denominator.

Case reports and “other” articles lowered journal IFs, and original articles and narrative reviews raised them in this study. Case reports, which attract readers among clinicians, who are often interested in their educational value, may be considered less important by researchers and therefore not be cited. For these reasons, editorial strategies that focus on raising an IF can lead to loss of content of the type clinicians want to read.17 Although case reports do not always serve to drive change in clinical practice, some of them provide important information that serves as a starting point for a hypothesis and further research.18

Dermatology provides good examples of this tie between cases and research. We have seen long-lived diagnostic entities emerge from descriptions of case series. Examples are toxic epidermal necrolysis and erythropoietic protoporphyria. Similarly, reports from series of cases of dermatitis herpetiformis included the observation of villous atrophy, leading to the unforeseen establishment of a relation to gluten intolerance. Single case reports have also furthered the advance of therapies in dermatology by suggesting new uses for treatments and by documenting unforeseen adverse effects.19

We share the opinion of Ruano et al8 that editorial boards should strike a balance between the need to increase IF and the need to maintain the attention of clinician readers.

The authors declare that they have no conflicts of interest.