NEW INFORMATION TECHNOLOGIES FOR ANALYSIS SKELETON PLANAR SCINTIGRAMMS OF PATIENTS WITH BREAST CANCER
- Authors: Kang X.1, Kosykh N.E.2, Levkova E.A.3, Razuvaev V.A.4, Savin S.Z.5
- 3rd Affiliated Hospital (Tumor Hospital) of Harbin Medical University
- The Far-Eastern State Medical University
- Medical Clinic Immunorehabilitation Center
- Far Eastern State Transport University
- Federal State Budget Educational Institution of Higher Education Pacific National University
- Issue: Vol 23, No 2 (2019)
- Pages: 187-196
- Section: ONCOLOGY
- URL: https://journals.rudn.ru/medicine/article/view/21359
- DOI: https://doi.org/10.22363/2313-0245-2019-23-2-187-196
In work is described practical approach to the expert system building for the analysis skeleton planar scintigramms. The aim is to analyze the numerical characteristics of bone metastases by scintigraphy. Objective. Progress in the development of bioinformatics and mathematical methods in biomedicine, as well as the development of computer and telecommunications systems and networks determines the look of the present and future of oncology technology and of medicine in general. At last years of one of the directions of high-tech-medicine development is a processing the digital image: improvement of quality of image, recovering image, its recognition of separate elements. Recognition of pathological processes is one of the most important problems of processing the medical image. Methods and results. Method of computer-aided analysis of planar osteostsintigrammy studied the skeleton of patients with breast cancer are in complete remission and in the phase progression of the disease with metastases to the skeleton. As analyzed parameter was used brightness of images. The study of the physiological accumulation of radiopharmaceuticals in patients without metastasis to the skeleton indicates a wide variation in the brightness values of the scintigram in some areas of the skeleton. At the same anatomical areas of the skeleton there are significant differences in the values of the index of average brightness. In almost all areas of the skeleton averages of the brightness lesions hyperfixation RFP for scintigram significantly prevail over those of «physiological» lesions hyperfixation. Thus, there is a direct relationship between the levels of accumulation of the radiopharmaceutical in areas of the skeleton without metastatic lesion and bone metastases occurring in these zones. Consider methodological approaches to studies of quality of qualifier at the expert system building for the analysis skeleton planar scintigramms, as well as results of conducting calculations.
Progress in the development of bioinformatics and mathematical methods in biomedicine, as well as the development of computer and telecommunications systems and networks determines the look of the present and future of medical technology and of medicine in general . At last years of one of the directions of development of cloud computing technologies in high-tech-medicine is a processing the digital image: improvement of quality of image, recovering image, its recognition of separate elements. Recognition of pathological processes is one of the most important problems of processing the medical image. By now, a number of standards for medical image have been developed [2-5]. By analogy with CAD/CAM systems (computer aided design and computer aided manufacturing) for technical applications, CAD (computer-aided diagnosis) systems are being developed for medical purposes [6-11]. Some of them are already successfully operating, but to date these systems are only «assistants» of a diagnostics who takes decisions. Herewith chosen objects are a reflecting a pathological process in the organism, but their categorization answers a question - all chosen objects are a manifestation of pathological process. As methods of parametric chosen object description is use texture, hystogram and morphometrical analysis, however combination of these methods is use seldom. CAD algorithms for medical image systems typically include image segmentation, the selection of some objects of interest («masses»), their analysis, parametric description of the selected objects and their classification. The classification of objects of interest may be made by the method of neural networks, support vectors, discriminant analysis [2, 12-14]. CAD systems significantly increase the effectiveness of radiographic diagnosis methods. However, the practical application of nuclear medicine demonstrates the continued information deficiency of the algorithms and programs that provide visualization and analysis of medical images. This is especially noticeable when the radionuclides, which are used, do not have a high capacity for accumulation in the pathological area. These include the bone scintigraphy method. Whole-body planar scintigraphy with 99mc-labeled phosphate compounds is widely used in the diagnosis of skeletal metastatic disease. Combining with crystals of hydroappatite, phosphate compounds reveal tumor-associated osteoneogenesis and can detect bone metastases long before the demineralization and bone destruction is detected by the X-ray examination [1, 15]. In addition, various pathological processes in the skeleton, as well as metastases, are revealed as polymorphic zones of hyperfixation (further referred to as PZH) of radiopharmaceutical preparations (further referred to as RPP). Despite the fact that bone scintigraphy is a method of choice in the early diagnosis of skeletal metastases of the tumors that tend to spread to the skeleton, the interpretation of bone scans in cancer patients is rather complicated. Categorization of objects of interest can be hold at the method neuronetworks, supporting vectors machine, discriminate analysis and others. Together with that, problem of automatic diagnostics of pathological processes as of medical image is distant from its permit. Actual problem stays a determination of optimum methods of parametric description of «objects of interest», that can render a direct influence upon the quality of categorizations given objects. This question is studied by means of developing us CAD-evaluation systems planar scintigrammes, skeleton beside tumor sick. Planar scintigraphy, skeleton is a method of diagnostics of bone metastasis. Essence of this diagnostic strategy is concluded in introduction sick radiopharmaceuticals (RP) - phosphate complexes, marked isotope 99mTc. Phosphate complexes are involved with crystals of hydroapatite, which level is raise in areas of bone metastasis. Gamma-radiation, stipulate by the isotope 99mTc, is register by sensors a gamma-camera. To the account this is form electronic scene of skeleton (anteroom and back projection), on which possible reveal a steady-state defeat long before the appearance denominated local demineralization and bone destruction, which is register at X-ray examination [3, 4, 8, 11, 16]. MATERIAL AND METHODS Early diagnosis of metastatic lesion of the skeleton has a significant influence on the choice of treatment of patients with malignant tumors. The material for our investigation was taken on Radiologic Departments of The Khabarovsk Regional Clinical Oncological Center. This training set included 169 front and 110 rear scans from 169 patients with malignant tumors. all patients received informed consent to participate in the study according to the Helsinki Declaration of the world medical Association[*]. Also had been protected patients from unlawful harmful effects on unauthorized access to information by a method of de-identification on the basis of anonymization of personal data pseudonymisation according DICOM rules . The study was approved by the ethics Committee of the Khabarovsk Regional Clinical Oncological Center. Therefore, there is a need to search for an approach to the optimization of scintigraphy analysis. The use of CAD analysis could substantially improve the specificity of osteoscintigraphy. At the same time work the on the application of CAD systems in the scintigraphy of the skeleton are rare and are not performed in our countries [13, 15]. The aim of the study was to determine the principles of optimizing the processing of planar osteostsintigraphy on the basis of CAD analysis. The results of scintigraphy are presented as plain (gray) digital images. In this case, a pixel image matrix can be the object of the analysis where each point (pixel) has a numerical value of brightness. Brightness value indirectly characterizes the intensity of the detected radiation (the amount of gamma photons identified by the detector) in each point of the object under study. The functional diagram CAD system that was developed included the following components: 1) image input; 2) segmentation of the image and storing it in the archive; 3) identifying diagnostic characteristics of the segmented images of the archive; 4) the formation of the training set and storing it in the archive; 5) the formation of the classifier; 6) the classification of new segmented areas of the image. In the study we used the scans of the patients with skeletal metastases of breast cancer, performed in a planar mode with Infinia-Hawkeye, the two-detector gamma camera manufactured by General Electric, using RFP pirfoteh-99mTc [1, 5]. The original image of the skeleton was recorded in the format of DICOM, which is central for the representation of medical images. Further analysis of the images was carried out in the program created in MATLAB . In this case, the original image is preserved in a matrix form in the shape of the internal MATLAB format . A significant role in the CAD image analysis was played by a segmentation method which helped to delineate the area corresponding to the skeleton. This segmentation was performed manually because to date there are no effective methods of automatic segmentation of complex anatomical objects. The mean value of image brightness was determined in the area corresponding to the skeleton, as well as the standard deviation (σ) of the value. It is a common point of view that the main feature of metastatic lesions on scintigraphy is a local hyperfixation of radiopharmaceutical preparation (RPP). The segmentation of RPP hyperfixation zones was conducted in the area corresponding to the skeleton and performed either automatically or manually. At the same time, the study adopted the hypothesis that pixels brightness values which were equal to or exceeded 2σ of the mean brightness value of the entire skeleton were considered as RPP hyperfixation which corresponded to a metastatic lesion. RESULTS The range of brightness values corresponding to metastases was programmatically specified. The areas with a given range were marked on the scans. The automatic segmentation which determines the RPP hyperfixation areas was made by the «threshold» method [2, 9]. In addition, the lesions suggestive of metastases in which the brightness value was less than 2σ were manually segmented. Identifying diagnostic features (characteristics) of segmented images is an important part of the functional scheme of the CAD system that was developed. Histogram, morphometric and texture parameters are calculated in the system by the methods of Haralik [20, 21] and local binary texture [2, 14, 22]. A group of histogram parameters includes the mean brightness value of the image, the standard deviation of the frightness from its mean value (σ), smoothness, the third moment of brightness, uniformity and entropy in the areas of RPP hyperfixation of the images [4, 10, 11, 13, 17]. The second group of symptoms is characterized by spatial texture brightness in PZH (polymorphic zones of hyperfixation). A so-called statistical approach, first proposed by R.M. Haralik in  was used. This approach makes it possible to create texture characteristics taking into account the distribution of brightness in the neighboring pixels of PZH [4, 6, 9, 11]. Textural characteristics are calculated on the basis of conjugate brightness level of an image in the local rectangular box and stored in a matrix of conjugate brightness levels, Grey Level Cooccurence Matrix (GLCM), according to the principle described in [13, 14, 16]. The indicators included in this group of features are represented by the dispersion of mean values used in the X and Y axes of digital image matrix, contrast ratios, autocorrelation, correlation, heterogeneity, homogeneity, energy, entropy, maximum likelihood, reciprocal difference and so on. The third group includes the texture parameters determined by the method of local binary texture [11, 14, 22]. The fourth group consists of the following morphometric parameters: the ellipsoidal coefficients of PZH, compactness and eccentricity of PZH [5, 9, 15]. After image segmentation isolated areas of PZH are analyzed by experts and are divided by them into metastatic lesions and non-metastatic areas. At the next stage the training set consisting of PZH objects is analyzed by an expert. Calculated histogram, texture and morphometric parameters corresponded to each object in the training set [2, 11, 12, 20]. The training set included 169 front and 110 rear scans showing the distribution of radiopharmaceutical in the front and rear parts of the skeleton. The data on the RPP hyperfixation areas, the numerical values of which were included in the training set are presented in Table. Table / Таблица The training set for metastatic and non-metastatic areas of radiopharmaceutical hyperfixation / Обучающая выборка метастатических и неметастатических областей гиперфиксации радиофармпрепарата Scanning plane / Плоскость сканирования The areas of radiopharmaceutical hyperfixation / Области гиперфиксации радиофармпрепарата Metastatic метастатические Non-metastatic неметастатические Front / Передняя 432 1 185 Back / задняя 223 734 Fig. Planar bone scintigrams of patient N: a - Segmentation of the skeleton; b - manual segmentation of RPP hyperfixation areas; c - automatically segmentation of RPP hyperfixation areas. Рис. Планарные сцинтиграммы костной системы пациента Н: а - сегментация скелета; б - ручная сегментация областей гиперфиксации радиофармпрепарата; в - автоматическая сегментация областей гиперфиксации радиофармпрепарата As an algorithm for the recognition of PZH (dividing them into metastatic and non-metastatic) the method of support vectors is used. It helps to create a classifier based on the training set (classifier function). Separate classifiers for the objects of front and back scans are also created. The evaluation of the quality of the classifier is carried out using ROC-curve expressing the ratio of true and false findings . The result of checking the effectiveness of the classifier on a test set of RPP hyperfixation areas using ROC-analysis is presented in [18, 22]. The algorithm of the unit for recognizing the new objects of PZH involves a sequential completion of the following stages: image segmentation (1), the calculation of diagnostic features for a new sample of objects (2), the use of the classifier to the selected sample of objects (3) and calculating the reliability of the classifier with respect to the sample (4). The result of the third phase of the program is the distribution of the objects of the analyzed image into two groups: metastatic and non-metastatic. The reliability of the classifier with respect to the sample was determined with the help of predicted variables classifier (PLVC), the method of calculation is presented in . PLVC values range from 0 to 1 . As an example, the analysis of patient N’s data is presented. Fig. represents the patient’s planar osteoscintigram in which a manual segmentation of the skeleton has been performed. At the same time, the closer the PLVC value, calculated for a particular object, is to 0 or 1, the more likely the object in question can be attributed to metastatic or non-metastatic. The results of recognizing RPP hyperfixation areas using an expert system (filling - «metastatic» lesions, no filling - nonneoplastic hyperfixation). Fig.c shows the segmentation of RPP hyperfixation areas that was performed automatically. The pixels, whose brightness values were equal to or exceeded 2σ of the average brightness of the entire skeleton, were treated as lesions suspicious for metastases. As can be seen in Fig. b, 10 areas of RPP have been segmented. To determine their nature in relation to the given sample the classifier, developed on the principle presented earlier, was used. In fact, the problem of image recognition has been solved with the help of the expert system, in which the expert’s ideas of visible signs of metastatic lesions were expressed in numerical values of histogram, texture and morphometric parameters. The recognition results are presented in Fig. b. As can be seen from the Fig. c, 7 out of 10 areas of RPP hyperfixation had signs of metastases. However, a more definite judgment on referring the areas to metastatic lesions can be made with the help of PLVC analysis. The CAD system under study has the functions of expert analysis and is based on the principles of image recognition. This is very important for the analysis of bone scans, where pronounced polymorphism of RPP hyperfixation areas creates significant difficulties for the correct visual evaluation of the data obtained with the help of sceleton scintigraphy. STATISTICS Method of supporting vectors machines was assume as a basis shaping a qualifier (classifying functions), on the base of educate sample. At qualifier was created apart for objects of front and back scanogrammes. Quality of a get qualifier was test for the sample non-classify OHF front (196 hearths) and back (110 hearths) scanogrammes. If take aim such experimental evaluation was show qualifier production measurement i.e. its ability to take faithfull deciding (probability of categorizations) . Evaluation of quality of qualifier was conduct by means of ROC-curve (receiver operating characteristic), expressing correlation of level of faithfull and false finding. Herewith paid digital area factor under ROC-curve - AUC (Area Under Curve). Scale of values was use for interpreting the values AUC. Conduct study of influence texture, hystogram and morphometrical parameters on the quality of qualifier. For this taxonomic function was form up with provision for parameters calculated only one of four use methods. Evaluation of quality of qualifier was realized by means of ROC-curve and AUC. The qualifier created on the grounds of given morphometrical analysis of OHF RP, has an average quality level. Herewith are absent any denominated differences between qualifiers for scintigrammes front (AUCant = 0,63) and back (AUCpost = 0,66) projections. Situation, close to stated above, is observe and for the qualifier, created on the grounds of the method of local binary texture. Quality of qualifier for front projections scintigrammes is described by ROC-curve with AUCant = 0,73), but for back projections scintigrammes - ROC-curve with AUCpost = 0,74. Quality of qualifier increases when using for its creation given texture analysis on Haralik . This particularly noticeably in the qualifier for back projections scintigrammes (AUCpost = 0,88). Quality of qualifier for front projections scintigrammes several below (AUCant = 0,78). Similar data were received at the examination of qualifier, build on the grounds of hystogram analysis material. However the most informative is a qualifier, build on the grounds of given all four considered above types of analysis. According to conducting ROC-analysis, quality of qualifier for back projections scintigrammes above, than for front (AUCpost = 0,96 and AUCant = 0,86 accordingly). DISCUSSION In such CAD system the expert’s ideas of visual signs of metastatic lesions are expressed in numeric values of histogram, texture and morphometric parameters. This is the basis of objective classification of the analyzed images. The principal difference between the system under anasysis from the most well-known CAD system proposed by M. Sadik for the evaluation of skeleton scans , is the calculation of predicted latent variability of classifier (PLVC) for RPP hyperfixation areas on scans [7, 12]. Thus, not only those RPP hyperfixation areas that are definitely abnormal (in this case - metastatic) can be distinguished, but also those whose nature needs specifying through further research and monitoring. PLVC calculation may be important in the clinical use of the CAD system. Naturally, the established system requires in-depth lab testing on different categories of patients and healthy individuals. However, now we can conclude that its diagnostic capabilities are largely dependent on the qualifications of the expert. Another factor affecting the accuracy of the recognition of metastatic lesions in the skeleton, is the size of the training set, which can be increased significantly with time. Therefore, increasing the use of the system is one way to increase its diagnostic reliability. CAD-systems included a translation of medical image from DICOM format  in the ambience MATLAB  with the following automatic segmenting of expressing «threshold method» and conservation its in the archive. Result of segmentation was show separation of hearths («ochag») an hyperfixation (OHF) RP, which and were presented «areas of interest» [7, 9]. For given areas was conduct texture analysis on the method Haralik  and local binary texture (Local Binary Patterns - LBP), as well as hystogram and morphometrical analysis [6, 13]. Simultaneously chosen hearths an RPC hyperfixation were analysed by the expert and are divide in metastasis and non-metastasis hearths. Form educate sample, cost from objects - analyse by the expert of hearths an RP hyperfixation. Each object in the educate sample corresponded to a count set of texture, hystogram and morphometrical parameters. List of parameters of evaluation OHF RP, note on scintigrammes. Automatic recognition of RP hyperfixation with division them on metastasis and non-metastasis was conduct on the base of method of supporting vectors (support vector machines - SVM) - a set of algorithms of type «education with the teacher», using for problems of categorizations and regression analysis [10, 16]. Main idea of method of supporting vectors machines - a translation of source vectors in the space of more high dimensionality and searching for separate hyperplane with the maximum clearance in this space [13, 19]. Two parallel hyperplanes are build on both sides of hyperplane, separate our classes. Separate hyperplane will be a plane, maximized distance before two parallel hyperplanes. Algorithm works in the suggestion, that than more difference or distance between these parallel hyperplanes, that less will be an average qualifier mistake. CONCLUSION so САD-systems to date have found while else limited using in the analysis planar scintigrammes of skeleton. Somewhat this is connected with difficulties of parametric description reveal OHF RP. However known САD-systems basically use given texture analysis. Unlike them proposed by us diagnostic system includes data, received with the help of other methods of analysis. Herewith, in spite of that informative morphological analysis and is local-binary Textural analysis is non-great, their combination with texture analysis on Haralik and hystogram analysis greatly raises qualifier production. The CAD system can successfully analyze (to improve the efficiency of the analysis) planar skeleton scans on the basis of automated computer diagnosis by increasing the diagnostic value of bone scintigraphy method. Automatic diagnostics of RPP hyperfixation areas requires a classifier function based on numerical analysis of texture, morphometry and histogram parameters using the support vector method as a recognition algorithm. Conflict of interest We declare that we have no conflict of interest.
About the authors
Xinmei Kang3rd Affiliated Hospital (Tumor Hospital) of Harbin Medical University
Author for correspondence.
N. E. KosykhThe Far-Eastern State Medical University
E. A. LevkovaMedical Clinic Immunorehabilitation Center
V. A. RazuvaevFar Eastern State Transport University
S. Z. SavinFederal State Budget Educational Institution of Higher Education Pacific National University
- Pasha S.P., Ternovoy S.K. Radionuclide diagnostic. M.: GEOTAR-media, 2008. 204 p. (in Russian).
- The Medical Image Analysis and Machine Intelligence (MIAMI) Research Group. http://www.ece.uwaterloo.ca/ ~miami (accessed 19 october 2018).
- Sadik M. Bone scintigraphy. A new approach to improve diagnostic accuracy. University of Gothenburg, 2009. 44 p.
- Brunet-Imbault B. et al. A new anisotropy index on trabecular bone radiographic images using the fast Fourier transform. BMC Medical Imaging. 2005. Vol. 5: 4. P. 11-16.
- Kosykh N.E., Eremenko A.V., Savin S.Z. Assessment of prognostic factors considering the volume of skeletal metastasis in patients with disseminated prostate cancer. The Siberian Journal of Oncology. 2017. Т. 16. № 1. P. 39-44 (in Russian).
- Kosykh N.E., Gostuyshkin V.V., Potapova T.P., Savin S.Z., Eremenko A.V. A computer-assisted diagnostic method for diagnosis of skeletal metastases based on planar scintigraphy data. Far Eastern medical journal, 2013. N 2. P. 33-35 (in Russian).
- Kosykh N.E., Gostuyshkin V.V., Savin S.Z., Vorojztov I.V. Designing the systems of computer diagnostics of medical images. Proc. of The First Russia and Pacific Conference on Computer Technology and Applications (RPC 2010). Vladivostok, Russia. 6-9 Sept., 2010. 4 p.
- Kosykh N.E., Litvinov K.A., Kovalenko V.L., Eremenko A.V. CAD-analisys of planar ostheoscintigramms for calculating the amount of skeletal metastases. Far Eastern medical journal. 2013. N 3. P. 33-34. (in Russian).
- Kosykh N.E., Smagin S.I., Gostuyshkin V.V., Savin S.Z., Litvinov K.A. System of automatic computer analysis of medical images. Information technologies and computing systems. 2011. № 3. P. 52-60 (in Russian).
- Kovalenko V.L., Kosykh N.E., Savin S.Z., Gostyushkin V.V. Method of rising for effectiveness of computer automatic technology in radionuclear diagnostics. Physicians and IT. 2013. N 6. P. 48-52 (in Russian).
- Lejbkowicz I., Wiener F. et al. Bone Browser a decision-aid for a radiological diagnosis of bone tumor. Comput. Methods Programs Biomed. 2002. 67(2). P. 137-154.
- Obenauer S., Hermann K.P., Grabbe E. Applications and literature review of the ВI-RADS classification. Eur Radiol. 2005. P. 1027-1036.
- Ojala T., Pietikainen M., Maenpaa T. Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns. IEEE Trans. Pattern Analysis and Machine Intelligence. 2002. Vol. 24 (7). P. 971-987.
- Soh L., Tsatsoulis C. Texture Analysis of SAR Sea Ice Imagery Using Gray Level Co-Occurrence Matrices. IEEE Transactions on Geoscience and Remote Sensing. 1999. Vol. 37. No. 2. P. 713-729.
- Kosykh N.E., Kovalenko V.L, Savin S.Z., Potapova T.P., Litvinov K.A. Some aspects of studying the expressing of hearths hyperfixation of radiopharmaceuticals on osteostsintigramms by means of computer automatic analysis. Journal of Rentgenology and Radiology. 2016. 97 (2). P. 95-100. (in Russian).
- Doronicheva A.V., Savin S.Z. WEB-Technology for Medical Images Segmentation. 3rd Russian-Pacific Conference on Computer Technology and Applications (RPC). IEEE Date Added to IEEE Xplore: 8 October 2018. P. 1-6.
- Web-site of DICOM [E-resource]. URL: http://www.dicom.html (accessed 19 october 2018).
- Gonzales P., Woods Р., Eddins W. Numerical processing the expressing in MATLAB ambience. M.: Thehnosfera. 2006. 615 p.
- De Brabanter K., Karsmakers P., Ojeda F., Alzate C. LS-SVMlab Toolbox User’s Guide, http://www.esat.kuleuven.be/ sista/lssvmlab.
- Haralick R.M. Statistical and structured approaches to the description of textures. IEEE-5, 1979. P. 98-118.
- Haralick R.M., Shanmugam K., Dinstein I. Textural Features of Image Classification. IEEE Transactions on Systems, Man and Cybernetics. Vol. SMC-3, no. 6, Nov. 1973. P. 11-22.
- Metz C.E. Fundamentals of ROC Analysis. Handbook of Medical Imaging. Vol. 1. Physics and Psychophysics. Beutel J, Kundel HL, and Van Metter RL, eds. SPIE Press (Bellingham WA 2000), Chapter 15. P.751-769.