CC BY
31
Известия Саратовского университета. Новая серия. Серия: Математика. Механика. Информатика. 2022. Т. 22, вып. 2. С. 224-232
Izvestiya of Saratov University. Mathematics. Mechanics. Informatics, 2022, vol. 22, iss. 2,
pp. 224-232 https://mmi.sgu.ru
https://doi.org/10.18500/1816-9791-2022-22-2-224-232
Article
Possibilities of using computer vision for data analytics in medicine
Peter the Great Saint Petersburg Polytechnic University, 29 Polytechnicheskaya St., St. Petersburg 195251, Russia
Oksana Yu. Iliashenko, ioy120878@gmail.com, https://orcid.org/0000-0001-9292-7759, AuthorlD: 331642
Ekaterina L. Lukyanchenko, lukyanchenko@mail.ru, https://orcid.org/0000-0003-2738-5513, AuthorlD: 1090381
Abstract. This article discusses the possibilities of using artificial intelligence technologies, namely computer vision, in the field of medicine. The relevance of the topic is due to the growing burden on medical personnel and medical institutions due to an increase in the number of elderly people, an increase in the number of patients with chronic diseases, as well as unforeseen circumstances, such as the SARS-CoV-2 pandemic in 2019-2021. In addition, many medical institutions are interested in providing high-quality services, increasing loyalty, and increasing the number of regular patients, and therefore feel the need to introduce the latest technologies and follow strategic development trends. The article describes how the physician can use the solutions offered by artificial intelligence in the course of his work to obtain a more accurate diagnosis and save time spent on the patient's history review. The authors propose an IT and technological architecture of a medical organization that uses computer vision in its work, created on the basis of the IT and the technological architecture reference model of a medical organization. The architecture implies the use of cloud infrastructure and specialized software and provides for both the introduction of new types of equipment, for example, 3D cameras, imaging sensors, and the use of traditional equipment: an ultrasound machine, X-ray equipment, an MRI machine. Keywords: artificial intelligence, computer vision, enterprise architecture, IT architecture, medicine
For citation: Iliashenko O. Yu., Lukyanchenko E. L. Possibilities of using computer vision for data analytics in medicine. Izvestiya of Saratov University. Mathematics. Mechanics. Informatics, 2022, vol. 22, iss. 2, pp. 224-232. https://doi.org/10.18500/1816-9791-2022-22-2-224-232 This is an open access article distributed under the terms of Creative Commons Attribution 4.0 International License (CC-BY 4.0)
Научная статья УДК 004.09
Возможности применения компьютерного зрения для аналитики
данных в медицине
Санкт-Петербургский политехнический университет Петра Великого, Россия, 195251, г. Санкт-
Петербург, ул. Политехническая, д. 29
O. Yu. Iliashenko, E. L. Lukyanchenko
,0
О. Ю. Ильяшенко, Е. Л. Лукьянченко
Ильяшенко Оксана Юрьевна, кандидат педагогических наук, доцент Высшей школы бизнес-инжиниринга, https://orcid.org/0000-0001-9292-7759, AuthorID: 331642, ioy120878@gmail.com Лукьянченко Екатерина Леонидовна, магистрант Высшей школы бизнес-инжиниринга, https://orcid. org/0000-0003-2738-5513, AuthorID: 1090381, lukyanchenko@mail.ru
Аннотация. В статье рассматриваются возможности использования технологий искусственного интеллекта, а именно компьютерного зрения, в сфере медицины. Актуальность темы обусловлена растущей нагрузкой на медицинский персонал и на медицинские учреждения в связи с увеличением числа пожилых людей и пациентов с хроническими заболеваниями, а также непредвиденными обстоятельствами, как, например, пандемия коронавируса SARS-CoV-2 в 2019-2021 гг. Кроме того, многие медицинские учреждения заинтересованы в предоставлении услуг высокого качества, повышении лояльности и увеличении числа постоянных пациентов, в связи с чем ощущают необходимость во внедрении новейших технологий и хотят следовать трендам стратегического развития. В статье описывается, как лечащий врач может использовать предлагаемые искусственным интеллектом решения в процессе своей работы для получения более точного диагноза и экономии времени на ознакомление с анамнезом пациента. Предложена ИТ- и технологическая архитектура медицинской организации, использующей компьютерное зрение в своей работе, созданная на основе референтной модели. Архитектура подразумевает использование облачной инфраструктуры, специализированного ПО и предусматривает как внедрение новых типов оборудования, например 3D камер, датчиков визуализации, так и использование традиционного оборудования: аппарата УЗИ, рентген-оборудования, аппарата МРТ.
Ключевые слова: искусственный интеллект, компьютерное зрение,архитектура предприятия, ИТ-архитектура, медицина
Для цитирования: Iliashenko O. Yu., Lukyanchenko E. L. Possibilities of using computer vision for data analytics in medicine [Ильяшенко О. Ю., Лукьянченко Е. Л. Возможности применения компьютерного зрения для аналитики данных в медицине] // Известия Саратовского университета. Новая серия. Серия: Математика. Механика. Информатика. 2022. Т. 22, вып. 2. С. 224-232. https://doi.org/10.18500/1816-9791-2022-22-2-224-232 Статья опубликована на условиях лицензии Creative Commons Attribution 4.0 International (CC-BY 4.0)
Introduction
The concept of artificial intelligence (AI) has evolved since 1950 when Alan Turing described the use of computers to develop critical thinking and human decision-making. Currently, artificial intelligence is an integral part of the development of the digital world and is used in all spheres of activity from medicine to industry and production [1]. Initially, AI was represented by a number of simple algorithms such as "if this, then that", then the algorithms rapidly became more complex, mimicking the neural connections of the human brain. The result of the development of the concept is the appearance of such classes of artificial intelligence as machine learning, deep learning, natural language processing, and computer vision [2]:
- machine learning is a field of artificial intelligence, which implies the "training" of computer algorithms to make decisions on their own. Machine learning is the driving force behind the intelligence of modern systems. Thus, the system learns independently (learning without a teacher) or with reinforcement from the environment, analyzing and classifying large amounts of data to identify the detected patterns and use them for further insights;
- deep learning is represented by machine learning methods that allow training a system not with algorithms, but with representations. Deep learning has proven to be effective after the increase in computing power has allowed the creation of complex neural network architectures. The use of deep learning made it possible to solve problems using computer vision, speech recognition, machine translation, etc.;
- natural language processing is a field of AI used to convert a natural language into a format understandable for computer systems for further analysis. It is used for speech recognition; handwritten and typewritten notes recognition;
- computer vision is an area of AI that deals with the identification of video and photographic images by raw pixels. Computer vision allows performing tasks such as detecting, classifying, and interpreting various objects to categorize predefined analysis results [3].
The purpose of this article is to study the application of computer vision methods in the field of medicine: areas of use, advantages and possible difficulties; development of an IT model and technological architecture of a medical organization that uses computer vision in its work, based on the reference architecture model, as well as the workflow of the physician using the opinion of artificial intelligence to make medical decisions.
Materials and methods
This study is based on the concept of an enterprise architecture. Currently, there are several standards and methodologies for modeling an enterprise architecture, but one of the most commonly used is the TOGAF (The Open Group Architecture Framework) standard, developed by The Open Group consortium.
The TOGAF standard is based on the Architecture Development Method (ADM), which describes a cyclical, phased plan for developing an enterprise architecture. In this study, ADM phases C and D were used.
Also, the authors use in their work the reference model presented in [4]. The reference model describes the IT and technology architecture of a healthcare organization, following the concept of value-based, personalized medicine and Health 4.0.
Results
In the last decade, in the field of medicine the most urgent problem is appeared to be the insufficient provision of medical institutions with personnel: general practitioners, specialist doctors, and junior medical personnel. The current medical staff bears the burden associated with examination, admission, counseling, prescribing treatment, and maintaining medical records, which affects the quality and quantity of work performed by them [5]. In addition, the population of most countries is aging rapidly, which in the near future will mean an increase in the flow of patients and the demand for medical services and examinations.
Controversial yet important is the issue of medical error in the process of making decisions about the further treatment of the patient. Telemedicine is actively used in Russia and around the world, which makes it possible to quickly assemble a medical consultation of highly qualified specialists and conduct it in a remote format. Patients also have access to solutions that allow them to remotely consult with their doctor, automatically collect, store and process data on medical indicators, for example, blood pressure, pulse, blood sugar level, etc.
Currently, the possibilities of using AI and machine learning in medicine are beginning to actively develop. The use of technologies such as computer vision, deep learning, and natural language processing will make it possible to fully implement such concepts as personalized, value-based, and predictive medicine [6]. Machine learning algorithms can be used to improve the accuracy of diagnosis, the efficiency of medical personnel and increase patient flow, therapeutic follow-up and clinical operations, the experience of interaction between the patient, and the medical organization as a whole.
The use of the latest technologies in medical data analytics is relevant because the volume of medical data already available is large and continues to grow daily. Medical data include data from electronic and paper medical records, results of clinical studies, and visual results of studies: MRI, X-ray, ultrasound, ECG, etc. [7]. In addition to the variety of types and sources of medical data, it is also characterized by high requirements for data confidentiality and security, anonymity, and medical secrecy.
Until recently, the healthcare industry could rely on small open source datasets or data collected by commercial companies. However, the dynamics of data exchange and data availability are already affecting and will continue to influence the development and availability of the use of big data analytics and artificial intelligence technologies [8]. These changes are stimulating research in the field of computer vision in medicine, which is reflected in the growth of research in areas such as multimodal education that combines computer vision and language recognition; 3D vision used in intensive care units, as well as work with videos-activity recognition, understanding of the ongoing movements to detect adverse events and respond to them [9].
In general, the process of computer vision operation consists of three stages:
- image acquisition — the basic stage, which consists of obtaining a photo or video image from hardware, for example, cameras of various types;
- image processing — the stage at which software is used for image processing: quality improvement, compression, reduction or increase in size and noise removal. At this stage, primitive operations are performed, which allow faster and better image processing and obtain the necessary analysis results in the future;
- image analysis and decision-making — the most important stage at which image segmentation or object identification occurs, and then the obtained characteristics are transferred to machine learning algorithms or other artificial intelligence technologies.
Thus, computer vision can be implemented using traditional machine learning algorithms, for example, hidden Markov models, random forest or random tree algorithms, or deep learning algorithms — convolutional neural networks, recurrent neural networks, and multilayer perceptron [10]. Deep learning is preferred because it allows the processing of large amounts of data without sacrificing hardware performance.
In medical practice, computer vision based on deep learning algorithms can be successfully applied in such areas as:
- gastroenterology [1];
- pulmonology [3];
- cardiology;
- dermatology;
- ophthalmology;
- clinical and imaging studies;
- surgery, etc.
The workflow implemented by a medical institution that uses computer vision
algorithms in its activities may look like this (Fig. 1). AI algorithms at the current stage of their development and application in medical practice cannot be the final link in making medical decisions. This means that the physician must familiarize himself with the results of the algorithms and, on their basis, make a diagnosis and prescribe treatment for the patient. For example, a system developed by the Californian company Gauss Surgical allows to accurately assess a patient's blood loss during surgery or childbirth by visualizing bloodied medical supplies — surgical sponges. The algorithms are able to distinguish between blood and non-blood fluids and trigger alerts about bleeding and critical blood loss in a patient, which allows them respond as quickly as possible and start a blood transfusion [11]. Nevertheless, the final decision, in this case, is made by the operating doctor, and the observation of artificial intelligence algorithms is a tool in the work of medical personnel and not a decision-making tool.
Fig. 1. An example of a medical institution workflow improved by computer
vision algorithms
The use of computer vision technologies in the process of carrying out diagnostic procedures has a number of specific features, for example:
- the need to find a balance in the accuracy of the interpretation of the results — minimization of false warnings, but sufficient accuracy is needed, not to miss the detection of the desired element on the image;
- the complexity of the configuration of the IT infrastructure of the solution, especially in the case of multifunctional use;
- features of working with medical images: the problem of transparency when visualizing the location of body organs — special instructions are needed to determine the location of organs in the image; high image resolution, which can exceed 100 thousand square pixels and several gigabytes of memory;
- insufficient amount of data available for training algorithms in the field of computer vision in medicine — despite the increased interest in this topic and the growing number of works devoted to research in the field of computer vision, the amount of high-quality and balanced data available for evaluating various computer vision techniques is not sufficient yet.
The use of computer vision by a medical institution unambiguously affects its IT architecture [12]. The medical institution must be mature enough to implement deep learning concepts: have transparent business processes described, have sufficient computing power and technological equipment, have sufficient funding, etc.
To build the IT and technological architecture of a medical organization using computer vision in its activities, the authors will rely on the reference model of IT and technological architecture of a medical organization presented in the study [4]. The model is performed in Archi software using the ArchiMate language supported by the TOGAF standard, which is actively used by international companies and institutions.
The reference model (Fig. 2) at the technological architecture level represents a cluster of servers, medical equipment, and personal gadgets used by patients to monitor vital medical indicators.
Fig. 2. Reference model of IT and technological architecture of a medical organization [4]
The application layer of the reference model is represented by a set of information systems used by a medical organization [13]:
- ERP — used by the organization to manage all types of resources; financial, operational, strategic, and tactical planning; marketing activities; etc.;
- MIS is a medical information system, used in work with patients to automate document flow, maintain electronic medical records, store and analyze clinical trials, manage bed funds, etc.;
- BI system in a medical organization is used for advanced analytics, predictive analytics and forecasting, visualization of the results.
Based on the reference architecture from [4], the authors propose a model of IT and technological architecture of a medical organization that uses computer vision technologies (Fig. 3).
Fig. 3. Proposed model of IT and technological architecture for a medical organization using
computer vision technology in its activities
The architecture model proposed by the authors consists of the following elements depicted on the technology layer:
- a server cluster located in the cloud infrastructure. The choice was made in favor of cloud technologies since training and testing machine learning algorithms requires large amounts of data and infrastructure flexibility [14]. For a medical organization, it will be more profitable and more convenient to use cloud resources, rather than maintain its own hardware infrastructure;
- in the cloud infrastructure, there is a LAN network, in which servers for the operation of MIS and BI systems, and software for the implementation of computer vision are located. The ERP system is not reflected in this diagram, since it is not directly related to the use of machine learning in the situation considered by the authors. Nevertheless, the ERP system of a medical organization can and should be used for more efficient business and resource management;
- load Balancer, an application that balances the load on servers to increase their performance and fault tolerance, has been added to the diagram in the cloud infrastructure. In addition, since the organization has to deal with medical data, a secure VPN connection to avoid leakage of especially sensitive data is provided;
- the diagram shows medical equipment that provides visual information for the use of machine learning. The type of equipment depends on the requirements and work of the medical organization: it can be either an X-ray machine or an MRI machine that is already available in the clinic, or cameras and sensors that are not used in traditional procedures.
The application layer has undergone changes in the part of Medical Equipment Software — it is presented by the use of software for the implementation of computer vision technology. As the deep learning model is trained on the provided medical data,
it will not be able to access the server and the database, but offer its opinion to the attending physician directly using the software on the application layer.
Thus, the proposed architecture can be used to implement computer vision technology in the work of medical clinics if they are ready to incur financial, time, and resource costs. The use of computer vision will relieve the medical staff, providing them with more time for important actions; will increase the accuracy of diagnosis and treatment prescription, which will affect the patient's attitude and loyalty to this medical institution [15]. The use of machine learning algorithms will reduce mortality in intensive care units and predict the deterioration of the patient's condition. All these advantages are especially relevant in our time since we see the impact of the SARS-CoV-2 pandemic on healthcare institutions around the world: the load on bed funds and medical personnel at the peak of the incidence rate was so high that it was not possible to provide adequate and high-quality medical care.
Despite the fact that the use of computer vision in medicine is a promising direction, in the field of medical imaging, some difficulties hinder the scaling of such solutions [16]. Firstly, testing the concept and conducting research, and introducing it into the work of the clinic, is characterized by high financial and time costs and requires the involvement of highly qualified specialists, which not many medical institutions can do. In addition, collecting data for training and testing a system can be time-consuming and expensive, as a particularly large volume of medical data is required, which must be collected, stored, protected, and anonymized.
The introduction of a new medical practice requires mandatory validation and certification, which is also costly for a medical institution both in terms of time and finances. Finally, compliance with international and local laws, policies, and practices is required, which can be a deterrent to implementation. At the same time, integrating new services with traditional research, familiar to patients, can be expensive and difficult, and the likelihood of sabotaging and ignoring the latest services using AI technology by patients of a medical institution is very high.
Conclusion
In this paper, the main areas of artificial intelligence are considered, the prerequisites for the popularization of machine learning and computer vision in the field of medicine are determined, and the areas of application of computer vision in medicine are considered. The authors studied the reference model based on the source [4], and proposed a modified model of IT and technological architecture of a modern medical organization, using computer vision algorithms in its work. The article briefly described the advantages and possible difficulties that may prevent the introduction of computer vision into the main processes of hospitals.
References
1. Kaul V., Enslin S., Gross S. A. History of artificial intelligence in medicine. Gastrointestinal Endoscopy, 2020, vol. 92, iss. 4, pp. 807-812. https://doi.org/10.1016/j.gie.2020.06.040
2. The future of artificial intelligence in health care. Emerging applications of AI in health care. Deloitte. Available at: https://www2.deloitte.com/us/en/pages/life-sciences-and-health-care/articles/future-of-artificial-intelligence-in-health-care.html (accessed 7 September 2021).
3. Khemasuwan D., Sorensen J. S., Colt H. G. Artificial intelligence in pulmonary medicine: Computer vision, predictive model and COVID-19. European Respiratory Review, 2020, vol. 29, Art. 200181. https://doi.org/10.1183/16000617.0181-2020
4. Ilin I. V., Lepekhin A. A., Ershova A. S., Borremans A. D. IT and technological architecture
of healthcare organization. IOP Conference Series: Materials Science and Engineering, 2020, vol. 1001, Art. 012141. https:/doi.org/10.1088/1757-899X/1001/1/012141
5. Il'in I. V., Il'yashenko O. Yu., Il'yashenko V. M. Architectural approach to the medical organization development in a digitalized healthcare environment. Journal of Management Studies, 2019, vol. 5, no. 1, pp. 37-47 (in Russian).
6. Vodolazsky K. D., Ilin I. V. Organization of information interaction of the medical organization with customers and resource providers. Journal of Economy and Entrepreneur-ship, 2021, no. 3 (128), pp. 920-929 (in Russian). https://doi.org/10.34925/EIP.2021.128.3485
7. LeCun Y., Bengio Y., Hinton G. Deep learning. Nature, 2015, vol. 521, no. 7553, pp. 436444. https://doi.org/10.1038/nature14539
8. Amin S. U., Hossain M. S., Muhammad G., Alhussein M., Rahman M. A. Cognitive smart healthcare for pathology detection and monitoring. IEEE Access, 2019, vol. 7, pp. 10745-10753. https://doi.org/10.1109/ACCESS.2019.2891390
9. Dubgorn A., Svetunkov S., Borremans A. Features of the functioning of a geographically distributed medical organization in Russia. E3S Web of Conferences, 2020, vol. 217, Art. 06014. https://doi.org/10.1051/e3sconf/202021706014
10. Mahmoodpour M., Lobov A., Hayati S., Pastukhov A. An affordable deep learning-based solution to support pick and place robotic tasks. Instrumentation Engineering, Electronics and Telecommunications - 2019: Proceedings of the V International Forum (November 20-22, 2019, Izhevsk, Russian Federation). Kalashnikov Izhevsk State Technical University Publ., 2019, pp. 66-75. https://doi.org/10.22213/2658-3658-2019-66-75
11. Gauss Surgical. Available at: https://www.gausssurgical.com/ (accessed 7 September 2021).
12. Ilin I., Iliashenko O., Iliashenko V. An architectural approach to managing the digital transformation of a medical organization. In: T. Devezas, J. Leitao, A. Sarygulov, eds. The Economics of Digital Transformation. Studies on Entrepreneurship, Structural Change and Industrial Dynamics. Springer, Cham, 2021, pp. 227-249. https://doi.org/10.1007/ 978-3-030-59959-1_15
13. Ilyin I. V., Ilyashenko V. M. Formation of requirements for a reference architectural model for digital transformation of a medical organization. Scientific Bulletin of the Southern Institute of Management, 2018, vol. 4, pp. 82-88 (in Russian). https://doi.org/10.31775/ 2305-3100-2018-4-82-88
14. Iliashenko O., Lukianchenko E., Lohyeeta N. A selection approach to the criteria for evaluating cloud platforms for conducting IT projects. DTMIS '20: Proceedings of the International Scientific Conference — Digital Transformation on Manufacturing, Infrastructure and Service. New York, NY, USA, Association for Computing Machinery, 2020, Art. 21. https://doi.org/10.1145/3446434.3446445
15. Bhattad P., Jain V. Artificial intelligence in modern medicine — the evolving necessity of the present and role in transforming the future of medical care. Cureus, 2020, vol. 12, no. 5, Art. e8041. https://doi.org/10.7759/cureus.8041
16. Gao J., Yang Y., Lin P., Park D. S. Computer vision in healthcare applications. Journal of Healthcare Engineering, 2018, vol. 2018, Art. 5157020. https://doi.org/10.1155/2018/ 5157020
Поступила в редакцию / Received 25.11.2021 Принята к публикации / Accepted 27.12.2021 Опубликована / Published 31.05.2022
