|
 
 |
| ORIGINAL ARTICLE |
|
| Year : 2022 | Volume
: 8
| Issue : 1 | Page : 29 |
|
MSLO – Melanocytic skin lesion ontology
Karolina Szturo1, Jens Haueisen2, Lukasz Piatek3
1 Department of Systems Engineering, Faculty of Technical Sciences, University of Warmia and Mazury, Olsztyn, Poland
2 Institute of Biomedical Engineering and Informatics, Technical University of Ilmenau, Ilmenau, Germany
3 Institute of Biomedical Engineering and Informatics, Technical University of Ilmenau, Ilmenau, Germany; Department of Artificial Intelligence, University of Information Technology and Management in Rzeszów, Rzeszów, Poland
| Date of Submission | 12-Apr-2022 |
| Date of Decision | 26-Jun-2022 |
| Date of Acceptance | 27-Jun-2022 |
| Date of Web Publication | 07-Dec-2022 |
Correspondence Address:
Karolina Szturo
M. Oczapowskiego 11 Street, Olsztyn 10-719
Poland
 Source of Support: None, Conflict of Interest: None  | Check |
DOI: 10.4103/digm.digm_18_22
Background and Purpose: Malignant melanoma is a high-grade skin cancer with high feasibility to metastasize to both regional and distant sites when detected late. Therefore, it is crucial to diagnose this type of cancer at an early stage to ensure effective treatment. The identification of melanocytic lesions is a difficult issue, even for experienced experts. The current development of information technology, particularly related to image analysis and machine learning, is an opportunity to support the work of specialists and detect malignant melanoma more effectively. The aim of this work is to present a melanocytic skin lesion ontology (MLSO) structure, which serves as a basis for a melanoma diagnosis system and includes the formalization of the experts' and literature knowledge. Subjects and Methods: MLSO describes the most commonly used melanoma assessing strategies: Argenziano's (also known as the 7-point checklist), Menzies', and Stolz's (based on the ABCD rule) strategies as well as Chaos and Clues. Results: In this work, a case study was conducted on the description of a dermatoscopic digital image of a melanocytic skin nevus. The nevus was evaluated according to all of the strategies included in the MLSO, and inferences were made based on these strategies. The analyzed lesion was classified as a benign nevus since no malignancy was indicated by any of the applied strategies. Conclusion: Initial results indicate the usefulness of MLSO in diagnosing skin cancer. A significant advantage of MLSO is that it provides results obtained using four strategies. Therefore, the results are more objective and the possible errors may be avoided. The MLSO structure is still developing and will be implemented into an automated skin cancer diagnosis system.
Keywords: Dermatology, Machine learning, Melanoma, Ontology, Skin lesion
How to cite this article:
Szturo K, Haueisen J, Piatek L. MSLO – Melanocytic skin lesion ontology. Digit Med 2022;8:29 |
| Introduction | |  |
Recently, a great interest in using specialized computer techniques in the medicine domain could be noticed, primarily in the field of supporting diagnostic procedures. Namely, numerous articles were published describing systems supporting the recognition of various types of anomalies visible and/or analyzed within medical images. Due to the obtained achievements in this domain, it seems reasonable to use image analysis and artificial intelligence/machine learning (ML) algorithms to increase the efficiency of diagnosing malignant melanoma disease.
Malignant melanoma is a skin cancer that may arise from melanocytes – the pigment cells of the skin. This type of skin cancer is characterized by rapid growth and easy metastasis and can cause a high mortality rate for patients when diagnosed late. According to the World Cancer Research Fund website,[1] in 2020, nearly 324,635 people worldwide were diagnosed with skin cancer and over 57,000 died.[2] It is estimated that early diagnosis of skin cancer can result in cure in 93% of cases. Therefore, a fast and effective diagnosis is crucial for treating this type of skin cancer. The first step in diagnosis is always a visual analysis performed by the doctors. In suspicious cases, a biopsy is performed in the second step. In this paper, we consider the first step of the diagnosis. The analysis of melanocytic lesions performed by the doctors (both specialists and general practitioners) is subject to error. It can be noticed in [Table 1] that the accuracy of the lesion classification depends mainly on medical knowledge (e.g., general practitioners and dermatologists). Moreover, even experts reach only a specificity of about 60%. In relation to the above facts, there is a great necessity for supporting the classification of the mentioned melanocytic skin lesions performed by practitioners through computer-based systems.
As it was demonstrated in many applications, automatic (and/or semi-automatic) image analysis can support medical diagnoses made by the experts, and in consequence, improve the accuracy of such diagnoses. In other words, the implementation of image processing and analysis algorithms enables a diagnosis to be more objective and reliable.
Several publications that describe the application of ontologies to systems in the medical domain have been done in the last few years. In this work, we propose the use of ontology – as a formal representation of malignant melanoma knowledge[4] – which consists of a set of concepts representing (i) types of skin lesion characteristics, (ii) visual features, (iii) detection algorithms and methods, and (iv) relations between those elements.
The main goal of this work is to create a novel melanocytic skin lesion ontology (MSLO) to include the expert's knowledge of skin lesions and selected information collected from the scientific literature. The ontology is structured, presented in a human-readable form, and is easy to interpret and upgrade. Moreover, the ontology will be the basis of a specialized computer system supporting the classification of skin lesions, which should ensure reproducibility of classification.
| Subjects and Methods | | |
Melanocytic skin lesion ontology
The ontology was created for image interpretation and melanoma diagnosis. MSLO represents knowledge about skin lesions and methods of melanoma detection as well as the basis for a nevus classification system. The proposed ontology is described by concepts and relationships between them. The MLSO is supposed to lead to the diagnosis, i.e., determining whether the lesion is benign, malignant, or suspicious. Classes represent skin lesion types as well as classification methods. Individuals represent specific types of skin lesions and rules define specific kinds of links between concepts.
The tool used to build and edit the ontology is Protégé, a free, open-source ontology editor and a knowledge management system.[5] Protégé provides a graphic user interface for defining ontologies and also includes deductive classifiers for verifying model consistency and for inferring new information based on ontology analysis.
MSLO includes the most commonly used strategies for assessing melanocytic skin lesions, i.e., Menzies' strategy,[3] Argenziano's strategy (known better as the 7-point scale or 7-point checklist),[6] Stolz's strategy (based on the ABCD rule),[7] and pattern analysis strategies (so-called Chaos and Clues).[8],[9],[10] The structure of the ontology for each strategy is presented in [Figure 1], [Figure 2], [Figure 3], [Figure 4]. | Figure 1: Excerpt of the MLSO structure related to the Menzies' strategy. Lesion to be classified as a melanoma must show asymmetry of the lesion pattern, more than one color, and at least one of the positive features. Information in the yellow box means that the score for feature Multiple colors has a value of 1 for lesion with at least five colors. MLSO: Melanocytic skin lesion ontology.
Click here to view |
 | Figure 2: Excerpt of the MLSO structure related to Argenziano's strategy (7-point checklist method). To be classified as melanoma, the pigmented lesion must show at least one Major criteria feature and minor criteria (score above 3). MLSO: Melanocytic skin lesion ontology.
Click here to view |
 | Figure 3: Excerpt of the MLSO structure related to Stolz's strategy (ABCD rule). Classification of a lesion into one of four classes (melanoma malignant, suspicious nevus, blue nevus, and benign nevus) is based on the TDS value (TDS is calculated from the values of features A, B, C, and D multiplied with factors). MLSO: Melanocytic skin lesion ontology, TDS: Total dermatoscopy score.
Click here to view |
 | Figure 4: Excerpt of the MLSO structure related to the Chaos and Clues strategy. A nevus is considered suspicious when Chaos and at least one feature of Clue are present. MLSO: Melanocytic skin lesion ontology.
Click here to view |
Menzies' strategy
Menzies' strategy in [Figure 1] examines symmetry and the presence of certain features, which include bluish-white veil, multiple brown dots, pseudopods, radial streaks, scar-like depigmentation, peripheral black dots/globules, multiple colors, multiple blue or gray dots, and a wide pigmented grid. Menzies' strategy assumes that benign nevi are symmetric and of a single color. Features indicating the possibility of melanoma are characterized by an asymmetric lesion with more than one color and the presence of at least one of the above-listed characteristics.
Argenziano's strategy
Argenziano's strategy in [Figure 2] involves separating minor and major criteria on a conventional scoring scale. Major criteria are defined by the presence of an atypical pigment network, an atypical vascular pattern, and a blue-whitish veil in the lesion. However, minor criteria include irregular streaks (i.e., pseudopods, radial streaming), irregular pigmentation, irregular spots/globules, and regression structures. The presence of each feature identified as a major criterion results in a value of 2 being added to the test score, while a 1-point scale is used when the minor criteria are present. Argenziano's strategy assumes that a nevus scoring of 3 or more may be malignant melanoma.
Stolz's strategy
Stolz's strategy in [Figure 3] is another widely used strategy for evaluating melanocytic lesions. It is a modification of the strategy proposed by Braun-Falco et al.[7] and uses the so-called ABCD rule, evaluating the four main features of the nevus analyzed. That four features are:
A (<Asymmetry>) - As the result of evaluating the asymmetry of the nevus.
B (<Border>) - The result of evaluating the nature of its border.
C (<Color>) - The result of identifying the number of colors in the nevus (out of six allowed).
D (<Diversity of structure>) - The result of evaluating the number of structures (out of five allowed: branched streaks, globules, structure of homogeneous areas, pigment network, and pigment dots).
The individual elements of the ABCD rule are used to determine the total dermatoscopy score (TDS). The value of the TDS identifies the nature of the lesion, which may be classified as:
- Benign nevus (TDS < 4.75 and lack of the blue color)
- Blue nevus (TDS < 4.75 and presence of blue color)
- Suspicious nevus (4.76 ≤ TDS < 5.45)
- Malignant melanoma – nowadays one of the most dangerous cancers[11] (TDS > 5.45).
The value of the TDS parameter is calculated according to the relation:[7]
TDS = 1.3 × <Asymmetry> + 0.1 × <Border> + 0.5 × Σ<Color> + 0.5 × Σ<Diversity of structures>.
Chaos and Clues strategy
The Chaos and Clues strategy in [Figure 4] describes a lesion using several basic structures: lines, pseudopods, circles, dots, and clumps. Each of these elements can be part of a specific pattern. In this method, the pattern is described first, then the color, and the pattern.[12] We define chaos as asymmetry of structure or color and pattern as one of eight patterns of malignancy:[12],[13] gray or blue structure, eccentric structureless area, thick lines reticular or branched, black dots or clods – peripheral, lines radial or pseudopods – segmental, polymorphous vessels, lines parallel, ridges or chaotic and polygons.
When no chaos is visible or chaos is visible without clues, there is no need for intervention. Otherwise, a biopsy of the lesion should be performed.
[Figure 5] and [Figure 6] show the class hierarchy of the MSLO and examples of how the various subclasses relate to each other. | Figure 5: Graph representation of the MSLO to illustrate the complexity of the ontology structure. Solid blue lines represent the hierarchy, while the colored dotted lines represent the object properties defining relations between classes. MLSO: Melanocytic skin lesion ontology.
Click here to view |
 | Figure 6: A tree representation of MSLO to illustrate the complexity of the ontology structure. (a) Class hierarchy; (b) Object properties hierarchy; (c) Data properties hierarchy. MLSO: Melanocytic skin lesion ontology.
Click here to view |
Considering the formal definition of the ontology described by Grzelak,[14] as well as the concept of Abbes and Sellami,[15] we made the formal assumptions for the ontology. The MSLO can be represented in various forms. One of them is a formal representation, described as follows:
MSLO = {C, R}
where C is the set of MLSO concepts and R is the set of semantic rules between them.
C is composed of three sets of concepts, which are lesion types (CLT), detecting methods and algorithms (CM), and skin lesion features (CF):
C = {CLT, CM, CF}
CM is composed of more specific concepts which represent diagnostic methods such as Argenziano's method (also known as 7-point scored diagnosis), Chaos and Clues, Menzies' scoring method, and Stolz's strategy (also known as ABCD Rule).
CM = {CABCD, CArg, CMen, CCaC}
CM includes skin lesion features, such as Asymmetry (CA), Border (CB), Color (CC), and Differential structures (CD).
CM = {CA, CB, CC, CD}
The four concepts: a, b, c, and d, where aϵ CA, bϵ CB, cϵ CC, and dϵ CD, are described with concepts and semantic rules proving the existence of melanoma malignant corresponding to Rabcd = 3, suspicious nevus corresponding to Rabcd = 2, blue nevus corresponding to Rabcd = 1, and that describing benign nevus corresponds to Rabcd = 0 in [Figure 7]. | Figure 7: Excerpt of the formal description of the MSLO related to the ABCD rule. MLSO: Melanocytic skin lesion ontology.
Click here to view |
| Results | | |
Evaluation of melanocytic nevus with different strategies
We demonstrate the use of the four strategies in MLSO by evaluating a dermatoscopic image of a melanocytic lesion in [Figure 8]. The expert from our research group evaluated the nevus for the presence of features indicative of malignancy. Individual scores were assigned to attributes stored in the ontology.
At first, Stolz's strategy was used to estimate the lesion. In this case, the nevus was determined to be asymmetric about one symmetry axis, and therefore, a value of 1 was assigned for the <lesionAsymmetry> feature. The sharpness of the transition between the nevus and healthy skin in each octal portion of the nevus was then assessed. This transition could be sharp or blurred (not affecting the outcome), which gives the feature <lesionBorder> a numerical value from 0 to 8. For the evaluated lesion, a = 0 was assigned to the feature as there were no sharp transitions. The nevus pigmentation was evaluated and two colors were found: light brown and dark brown. Thus, the feature <lesionColor> was = 2. The last evaluated feature was the diversity of structure. The presence of three features was determined: pigment dots, pigment globules, and pigment network. Therefore, to the feature <lesionDiversityofstructure>, a = 3 was assigned. Finally, based on the TDS = 3.8, the analyzed case was estimated to be in the benign nevus class.
The detailed evaluation results for Stolz's strategy are presented in [Table 2]. | Table 2: The Stolz's strategy (i.e., ABCD rule) nevus evaluation results
Click here to view |
Regarding the evaluation of the nevus by Menzies' strategy, the expert assessed that the nevus was asymmetric, and two colors were present. Therefore, the features <asymmetryOfPattern> and <moreThanOneColor> were given values of 1 and 2. As no features from the “positive features” category were found, the decision regarding the classification of the lesion is benign nevus.
The features of the evaluation according to Menzies' strategy are included in [Table 3].
Argenziano's strategy was the third applied method. [Table 4] includes the evaluation elements according to this strategy. The expert assigned a value of 0 to all the features, indicating that the nevus is benign. | Table 4: Argenziano's strategy (7-point checklist) nevus evaluation results
Click here to view |
An assessment using the Chaos and Clues strategy was the final step. The nevus has features of Chaos, but the elements categorized as Clues are not visible. Hence, the decision of no intervention was indicated.
The results of the lesion assessment with the Chaos and Clues strategy are presented in [Table 5].
In addition, according to the expert's decision, a subjective conclusion was made that the lesion was classified as benign nevus, i.e., without any malignancy.
| Discussion | | |
In the past few years, ontology has been used extensively in expert systems. It enables the classification of images, image features, their interpretation, and the use of common relationships, including relationships between images, shapes, and specific interpretations.[16]
Existing ontologies are often used in medical systems, e.g., cell image analysis ontology,[17] ontology-driven image analysis for histopathological images,[18] ontology of mammography,[19] or geoinformatics/geodesy, e.g., ontology-driven geographic information systems.[20],[21] The usefulness of ontologies for classifying and indexing images is also noted in systems such as semantic image annotation,[22] PetroGrapher,[23] annotation of image segments with ontologies,[24] or satellite image search and sorting system.[25]
Ontology in image analysis systems allows to:
- Obtain detailed and precise classification results[26],[27]
- Easily extend to other components[20]
- Use existing ontologies and adapt them to the requirements of new ontologies[17]
- Obtain possibilities of data mining[28]
- Apply fuzzy logic to uncertain data.[29]
Three ontologies in systems supporting the diagnosis of skin cancer have been described in the literature. All these systems are intended to support skin lesions classification to specific classes based on digital images.
In the first publication, Laskaris et al.[30] proposed a system for describing skin lesion images based on a human-like perception. Unlike other existing works on the classification of skin lesions, the authors focused not only on melanoma but also on other types of skin cancer and benign lesions. The system gives images of skin lesions a semantic label in a way humans do. In the first part of the work, the authors tried to find out how users perceive each change. Then, the image description system was trained by ML. In the first part, five attributes of skin lesions were selected: color (from light to dark), uniformity of color (from uniform to nonuniform), symmetry (from symmetrical to asymmetrical), edges (from regular to irregular), and texture (from smooth to rough). Volunteers were asked to assign values to each attribute for each displayed change. In the second part, 93 features were extracted from each change and used an ML algorithm. The authors' results are promising, especially for color-related attributes, where more than 80% of lesions were classified in the same semantic classes classified by humans. Because the labels assigned to each image have a very large variation (meaning different people describe the same image differently), the resulting classification accuracy was quite low in some cases.
Maragoudakis and Maglogiannis[31] proposed an algorithm that enhances the original domain model using agglomeration clustering methods with distance criteria to the existing ontological structure. This solution enables better classification and retrieval of new categories of skin lesions. The whole process of creating a new dermatology ontology includes adding new features of skin images found on the web to the new structure based on the features of skin lesion images. The first results confirm the ability of the proposed algorithm to generate new ontologies from image data on the network and select the one that best fits the core ontology.
Abbes and Sellami[15] described an ontology-based semantic analysis of images of skin pathological changes. In developing the ontology, the authors collaborated with experts in the field of dermatology. The concept of ontology and semantic annotations were presented. Low-level features describing the shape, color, and texture of the skin lesion were distinguished. A Bag of Words (or natural language model) model was generated to describe these characteristics using an ML classifier support vector machine. An important step in semantic analysis is to define rules relating to different concepts. Decision-making rules were produced using the ABCD rule. A comparison of lesion images (based on 206 public images) showed that ontology offers an effective framework of analysis in which the semantic relationships between concepts can support greater expert knowledge and can be suitable for classifying skin lesions with good accuracy (76.9%).
It was found that the specialist dermatoscopic systems described in the literature most commonly use the ABCD rule to detect melanocytic nevi of the skin. MLSO is more complete and includes four strategies (Stolz's, Menzies', Argenziano's, and Chaos and Clues) to make the results more stable as the classification obtained by applying different strategies may provide different results.
A limitation of our work lies in the restriction of the available data set. Only adult data were available. An extension to data from children is desirable and subject to future research.
The aim of developing the MLSO was to represent and formalize the knowledge about melanoma skin lesions obtained from experts and professional literature. In future work, the application of ontology in a skin cancer diagnosis system, in addition to facilitating access to knowledge, may result in repeatability of classification results. The constructed ontology is readable for users as well as interpretable and modifiable.
Financial support and sponsorship
This study was supported by grants from Polish National Centre for Research and Development (Grant Number WPN-3/3/2019/DigiSkinDia “Digital solutions for automatic skin cancer diagnosis”) and German Federal Ministry of Education and Research (Grant Number 01DS19012A).
Conflicts of interest
There are no conflicts of interest.
| References | | |
| 1. | |
| 2. | |
| 3. | Przystalski K. Detection and classification of pigmented skin lesions on multilayer images. Polska Akademia Nauk 2014.  |
| 4. | |
| 5. | Rocha R, Araújo A, Cordeiro D, Ximenes A, Teixeira J, Silva G, et al. DKDOnto: An ontology to support software development with distributed teams. Proc Comput Sci 2018;126:373-82. |
| 6. | Argenziano G, Zalaudek I, Ferrara G, Johr R, Langford D, Puig S, et al. Dermoscopy features of melanoma incognito: Indications for biopsy. J Am Acad Dermatol 2007;56:508-13. |
| 7. | Braun-Falco O, Stolz W, Bilek P, Merkle T, Landthaler M. The dermatoscope ― A simplification of skin surface microscopy of pigmented skin lesions. Hautarzt 1990;41:131-6. |
| 8. | Argenziano G, Moscarella E, Piccolo V, Agozzino M, Longo C, Pellacani G. Color Atlas of Dermoscopy. Padova: Piccin Nuova Libraria S.p.A.; 2020. |
| 9. | Tschandl P, Cliff Rosendahl. Dermatoscopy – Chaos and Clues. Colour Atlas Dermoscopy. 1 st ed. New Delhi: The Health Sciences Publisher; 2017. p. 339-44. |
| 10. | Michalska M. Analyses of skin lesion areas after thresholding. Inform Autom Pomiary Gospod Ochr Środowiska 2020;10:9-12. |
| 11. | Topik E. WHO Melanoma Programme. Diagnosis And Clinical Treatment of Melanoma of the Skin. Warsaw: PFESO; 1998. |
| 12. | Cameron A, Kittler H, Rosendahl C, Tschandl P. Dermatoscopy: An algorithmic method based on pattern analysis. Gdańsk: Via Medica; 2012. |
| 13. | |
| 14. | Grzelak W. Ontology − An Attempt to Systematize Concepts. Bus Inform 2013;4:159-68. |
| 15. | Abbes W, Sellami D. Automatic skin lesions classification using ontology-based semantic analysis of optical standard images. Procedia Comput Sci 2017;112:2096-105. |
| 16. | Gruber TR. A translation approach to portable ontology specifications. Knowl Acquis 1993;5:199-220. |
| 17. | Colantonio S, Martinelli M, Salvetti O, Gurevich IB, Trusova YO. Cell image analysis ontology. Pattern Recognit Image Anal 2008;18:332-41. |
| 18. | Othmani A, Meziat C, Loménie N. Ontology-driven image analysis for histopathological images. Lect Notes Comput Sci 2010;6453:1-12. |
| 19. | Przelaskowski A, Podsiadły-Marczykowska T, Boniński P, Wróblewska A. Integrated system for assistance of mammography diagnosis. Elektron Konstr Technol Zastos 2008;49:122-7. |
| 20. | Fonseca FT, Egenhofer MJ, Agouris P, Cmara G. Using ontologies for integrated geographic information systems. Trans GIS 2002;6:231-57. |
| 21. | Fonseca FT, Egenhofer MJ. Ontology-driven geographic information systems. Proceedings of the 7 th ACM Symposium on Advances in Geographic Information Systems; 1999. [doi: 10.1145/320134.320137]. |
| 22. | Koletsis P, Petrakis EG. SIA: Semantic image annotation using ontologies and image content analysis. Lect Notes Comput Sci 2010;6111:374-83. |
| 23. | |
| 24. | Lingutla NT, Preece J, Todorovic S, Cooper L, Moore L, Jaiswal P. AISO: Annotation of image segments with ontologies. J Biomed Semantics 2014;5:1-6. |
| 25. | Almendros-Jiménez JM, Piedra JA, Cantón M. An ontology-based modeling of an ocean satellite image retrieval system. Proceedings of the 2010 IEEE International Geoscience and Remote Sensing Symposium; 2010. p. 2237-2240. [doi: 10.1109/IGARSS.2010.5649457]. |
| 26. | Breen C, Khan L, Ponnusamy A. Image Classification Using Neural Networks and Ontologies. Proceedings International Workshop on Database and Expert Systems Applications DEXA 2002; 2002 January. p. 98-102. [doi: 10.1109/DEXA.2002.1045883]. |
| 27. | Nattawuttisit S, Usanavasin S. Information retrieval by image recognition using ontology mapping with neural networks. Int J Comput Sci Electron Eng 2013;1:1-5. |
| 28. | Gao Y. Ontology-Based Large-Scale Image Classification, Indexing and Exploration; 2007. |
| 29. | Elleuch N, Zarka M, Ammar A Ben, Alimi AM. A Fuzzy Ontology – Based Framework for Reasoning in Visual Video Content Analysis and Indexing. Proceedings 11 th Int Work Multimed Data Mining, MDMKDD'11 – Held Conjunction with SIGKDD'11; 2011. p. 1-8. [doi: 10.1145/2237827.2237828]. |
| 30. | Laskaris N, Ballerini L, Fisher RB, Aldridge B, Rees J. Fuzzy Description of Skin Lesions. Vol. 7627. Med Imaging 2010 Image Perception, Obs Performance, Technol Assess; 2010. p. 762717. [doi: 10.1117/12.845294]. |
| 31. | Maragoudakis M, Maglogiannis I. A medical ontology for intelligent web-based skin lesions image retrieval. Health Informatics J 2011;17:140-57. |
[Figure 1], [Figure 2], [Figure 3], [Figure 4], [Figure 5], [Figure 6], [Figure 7], [Figure 8]
[Table 1], [Table 2], [Table 3], [Table 4], [Table 5]
|
Search Pubmed for