A growing number of patients, physicians, and institutions are using telemedicine services,1 and the world telemedicine market is expected to grow from US$9.8 billion in 2010 to some $US27.3 billion in 2016.2 Dermatology appears to be an ideal specialty for telemedicine.3 Indeed, it is the clinical specialty that currently accounts for the largest number of telemedicine studies.1 Teledermatology can be carried out in real time using videoconferencing systems (VC) or asynchronously using store and forward (SF) systems, a modality in which clinical images are obtained by the physician, stored, and then forwarded to the specialist for later assessment. VC requires greater coordination and consumes more time and resources.4,5 It can also be more costly than conventional face-to-face (FF) consultation.6 While VC allows the specialist to interact directly with the patient, the reliability of the method (as measured by diagnostic agreement) can be similar to that of SF teleconsultation.4,7,8 Thus, medical professionals and healthcare providers increasingly consider SF to be the most cost effective and convenient system.9

The teleconsultation is a complex intervention that depends on multiple interactions and is influenced by local administrative and organizational variations. As a result, it is difficult in telemedicine to conduct studies that provide a high level of evidence, such as randomized clinical trials. A systematic review concluded that teledermatology is far from being a mature technology,10 and that more randomized trials involving real patients are needed to provide high quality evidence.

To date, most of the research in teledermatology has assessed reliability on the basis of concurrence in diagnosis.11 Most of these studies distinguish between complete agreement (concurrence on a single diagnosis) and aggregate agreement (concurrence on at least one diagnosis when a differential diagnosis is proposed) (Table 1). Although it can be high, diagnostic reliability does vary considerably, with the percentage of complete agreement ranging from 47% to 90% and for aggregate agreement from 60% to 99%.1,11 While many authors have studied the diagnostic reliability of SF, none of the studies have been randomized controlled trials and almost all were conducted in an experimental rather than a clinical setting.10

In routine clinical practice, it is the primary care physician or other health professional who, in a remote location, typically a primary care center, sees the patient, records the clinical history, and takes the photographs. The clinical data is then forwarded through an intranet or the Internet to the hospital-based dermatologist for assessment. However, in nearly all of the published studies on the reliability of SF, the clinical history and the photographs were taken on site in the hospital itself (not in a remote location) and they were not taken by primary care physicians, but rather by experts, whether dermatologists, dermatology residents, professional photographers, or research assistants.11,12 This bias could artificially increase the apparent reliability of SF in these studies.

In fact there are no studies that demonstrate the reliability of SF in general skin disease in routine clinical practice. This was the main objective of the present study: to assess interobserver reliability in SF when the patients were assessed in a remote location under the conditions of routine clinical practice. A secondary objective was to investigate whether teledermatologists were able to predict their own diagnostic errors under these conditions.

In a routine clinical practice setting, the rate of complete and aggregate diagnostic agreement was 75% and 90%, respectively, in the intermodality comparison (SF vs FF) and 70% and 85%, respectively, in the intramodality comparison (SF vs SF). These findings are comparable to the highest concordance rates published in more than 40 studies on the reliability of SF and show that SF is very reliable in the clinical setting.5

While the accumulated evidence on this topic is of high quality, it may be weakened by certain types of bias that we have attempted to avoid our study.1,13 DERMATEL was a randomized prospective trial that included interobserver comparisons. The sample was representative of the general population as it included all types of disorders (not just tumors) and had an appropriate age and sex distribution. The sample used for SF comparisons was large—510 teleconsultations (170 × 3)—and, most importantly, the study took place in a routine clinical setting: 18 primary care physicians took photographs and recorded the clinical data remotely in health care centers. The teleconsultations were then transmitted digitally over the Internet. Thus, the characteristics of the study guarantee a high degree of internal and external validity.1

Most earlier studies have assessed the reliability of SF under ideal “laboratory” conditions, unlike those of routine workflow in a normal clinical setting.1,10,11 In the study by Krupinski et al.14 the photographs were taken by a medical student under the direction of a dermatologist, and in the study by Lyon et al.15 the photographs were taken by a resident in dermatology. In one study,16 the photographs were taken by a professional photographer, and in others by a research assistant.17,18 Finally, in a number of other studies, photographs were taken by a nurse trained in clinical photography7,19–21 In most of these studies, the images and clinical data were obtained on site in the hospital where they were later assessed by the ‘tele’ dermatologist.7,15–21 This procedure made it easier to carry out the study but may have introduced bias.

It has been observed that the results are poorer when several different primary care physicians are responsible for taking the photographs and collecting the clinical data and when the teleconsultations are sent from a remote location.12,22,23 Only 2 earlier SF studies were similar to DERMATEL in that they assessed all types of skin disease (tumors and other processes) and emulated the conditions of routine clinical practice.12,22 No valid conclusions can be drawn from one of these studies due to flaws in the study design.22 The other produced the worst results of the 41 studies on SF in the literature, reporting complete agreement of 54% and aggregate agreement of 63%.12,13 The authors of that study attributed the poor results obtained to their use of real conditions rather than the ideal conditions created in the other studies.12 Some studies focusing exclusively on skin tumors have achieved good reliability in a clinical setting, an outcome that supports our findings.24 An important objective of the present study was to demonstrate the diagnostic reliability of SF in a clinical setting and not only for tumors but also for skin disease in general.

The design of the DERMATEL study included the use of a predefined medical history form containing a clear set of items, some of which were mandatory. Moreover, the protocol specified that the primary care physicians had to receive training in digital photography. Data storage and processing was centralized in the hospital. All of these characteristics may explain why the results of our study tend to be better on average than those reported in the literature.

Intraobserver diagnostic agreement was very high in our study, with values comparable to those reported by the study in the literature reporting the best results for reliability.25 In addition, our findings relating to reliability (SF vs SF intramodality agreement) are similar to those published previously.7,14,16,17

In DERMATEL, diagnostic agreement correlated with the teledermatologists’ confidence in their own diagnosis and was associated with the recommendation that an in-person consultation was needed and, to a lesser extent, with the quality of the images and clinical data in the teleconsultation (Table 2). It would appear, therefore, that the teledermatologist can evaluate the likelihood of an error in diagnosis and avoid this outcome by recommending a conventional in-person consultation. Although the correlation between diagnostic agreement and diagnostic confidence has been studied extensively in the literature,7,14,18 the present study was the first to analyze the correlation between diagnostic agreement and the decision to recommend an in-person consultation.4

Our statistical analysis showed with a high degree of statistical significance that the teledermatologist can predict and therefore avoid errors in diagnosis (P < .001 and χ2 > 34.9) (Table 4). If the teledermatologist considers the quality of the images and clinical history to be adequate and has a high degree of confidence in his or her own diagnosis, the likelihood of an error is very low. In cases in which the quality of the images or of the clinical history is rated as inadequate or diagnostic confidence is low, the dermatologist can prevent any error in diagnosis by recommending further assessment in an FF consultation.

The results of DERMATEL also showed that the dermatologist, as an expert morphologist, can easily make a diagnosis when the case is typical and the image provided is of good quality. Logically, if the case is atypical or complex (e.g., autoimmune disease, severe psoriasis, or contact dermatitis), the dermatologist must see the patient in person at the clinic.

The error analysis showed that very diverse disorders (Table 6) can be diagnosed with a high degree of reliability. The cases assessed included a wide variety of disorders: common conditions such as common warts and melanocytic nevae; rare diseases, such as superficial actinic porokeratosis and solitary mastocytoma; and potentially serious disorders, including nonmelanoma skin cancer and bullous disorders. The type of cases that gave rise to disagreements illustrate some of the limitations of SF (Table 5). As has been suggested previously, lesions located in the mouth (case 5) and on the scalp (case 6) are more difficult to diagnose in a teleconsultation because they are more difficult to photograph.26 Similarly, the diagnosis of nodules (case 1) poses problems because of the importance of palpation in their assessment. In all the cases where there were important differences of opinion, the photographs were of poor quality and the dermatologists recommended in-person consultation.

Many skin conditions can be managed on-line, thus avoiding the need for in-person consultations. In other cases, despite a correct diagnosis, an FF consultation is necessary for the management of the condition, whether because of the need for diagnostic tests or therapeutic procedures (Table 3). SF can only eliminate the need for an in-person consultation in 50% of cases.27 Online presurgical planning may eliminate the need for a further 15% of live consultations when patients can go directly to a surgical intervention following a teleconsultation.28

The DERMATEL study was completed in 2007. Since then our group has developed a software package (DERCAM) that integrates digital photographic images into the radiology picture archiving and communication system (PACS) and facilitates SF teledermatology. Since this system was introduced in our hospital in 2009, 4 dermatologists have seen over 2000 patients and an FF consultation has been avoided in 53% of these cases. During this period we have continued to assess both the quality of the photographs and the dermatologist's confidence in his or her diagnosis. The results obtained in a real clinical setting have been comparable to the experimental results of the DERMATEL study: high diagnostic confidence in 74% of cases and good photographic quality in 81%.

A systematic review of the literature revealed the need for more quality research in teledermatology to provide quality evidence supporting the use of this technology in routine clinical practice.10 We believe that our study and other recent studies provide evidence to support the usefulness of teledermatology in routine practice.29–32 One randomized clinical trial showed that SF can be a cost-saving strategy for providing dermatological care,30 producing clinical outcomes similar to those obtained with conventional consultations.31 A recent study demonstrated the feasibility of SF on a large scale29: 1820 general practitioners and 166 dermatologists performed more than 37 000 SF consultations between 2007 and 2010. There was no need for an FF consultation in 74% of these cases. The authors of that study concluded that SF should be considered a possible pathway of referral from primary to secondary care and we agree with this conclusion.

The diagnostic validity of SF for skin tumors compared to the gold standard of histologic study is very high, but that of FF consultation in such cases is even higher.5 The DERMATEL study has shown that teledermatologists can predict and prevent errors by recommending a live consultation when they have any doubts. For example, an analysis of the literature demonstrates the greater diagnostic ability of FF consultation vs teledermatology in the case of pigmented lesions,33 but it has recently been shown that the routine use of SF in a practice setting can enhance early diagnosis of malignant melanoma because the teledermatology procedure involves more active collaboration on the part of the primary care physicians.32 Teledermatology can also offer other advantages over in-person consultation: greater accessibility, shorter time frame, lower costs, reduced patient travel, and enhanced training for primary care physicians. Some of these advantages have already been demonstrated, while other have not yet been proved. Thus, more studies are needed to confirm the health care benefits of teledermatology.1,5

A limitation of the present study, in our opinion, is that we only have histologic confirmation for patients with malignant tumors, who were a minority. However, a histologic study was performed when there was diagnostic doubt, whether the condition was inflammatory or neoplastic. These studies served to clarify the doubts (Table 5). Nonetheless, particularly in the case of common diseases not involving a tumor, such as seborrheic dermatitis or acne, the gold standard for diagnosis in clinical practice is expert opinion rather than histology. Thus, one limitation of our study and many others is the absence of a valid measurement of interobserver reliability between dermatologists in FF consultation. This is a crucial factor in measuring diagnostic reliability, since it is possible that the interobserver disagreements (between the teledermatologist and the dermatologist who saw the patient in person) are not the result of the modalities used (teleconsultation vs FF consultation) but rather interobserver variability. The only 2 studies that included such a measurement reported widely differing results.17,34 While many authors have drawn attention to this problem,1,12,17,35 surprisingly little research has been undertaken to resolve the issue.

In conclusion, our study showed SF to be reliable in a real clinical setting. The complete independence of asynchronous SF teledermatology from the constraints of time and place make it an enormously attractive option. Our findings underscore the importance of using a standardized medical history form including mandatory items, and the need to train the physicians in digital photography. Logically, the dermatologists must be aware of the potential seriousness of the disease they are assessing. Whenever the photograph is of poor quality or their confidence in the accuracy of their own diagnosis is low, dermatologists should recommend an in-person consultation or a repeat teleconsultation. High quality evidence supporting the routine use of teledermatology as an alternative to the conventional pathways for referral of patients to specialized care has started to accumulate gradually.

The authors declare that they have no conflict of interest.

This study received funding through the Programa de Investigación Biomédica y en Ciencias de la Salud from the Spanish Ministry of Health and Consumer Affairs Health Research Fund (Fondo de Investigación Sanitaria) under the registration number PI021695, cofinanced by FEDER funds. It also received financial support from the Fundación para la Investigación Sanitaria en Castilla-La Mancha (FISCAM).