CC BY
9ОБЗОРНЫЕ СТАТЬИ/ ADABIYOTLAR SHARHI/ REVIEW ARTICLES
Дарья Хеммерлинг,
Университет науки и технологий, факультет электротехники инженерии, автоматики, информатики и биомедицинской инженерии, Краков, Польша Бенедетта Синьорелли, Кафедра наук о здоровье человека, факультет медицины и хирургии, Университет Флоренции, Италия Войцех Вояковски, Отделение кардиологии и структурных болезней сердца, Медицинский университет Силезии, Катовице, Польша
Михал Тендера,
Отделение кардиологии и структурных болезней сердца, Медицинский университет Силезии, Катовице, Польша
Томаш Ядчик
Отделение кардиологии и структурных болезней сердца, Медицинский университет Силезии, Катовице, Польша, Международный центр клинических исследований, Университетская клиника Святой Анны Брно, Брно, Чехия
ГОЛОСОВЫЕ ТЕХНОЛОГИИ ПРИ СЕРДЕЧНО-СОСУДИСТЫХ ЗАБОЛЕВАНИЯХ
For citation: Daria Hemmerling, Benedetta Signorelli, Wojciech Wojakowski, Michal Tendera, Tomasz Jadczyk. Voice technology in cardiovascular diseases. Journal of cardiorespiratory research. 2021, vol. 2, issue 4, pp. 9-12
http://dx.doi.org/10.26739/2181-0974-2021-4-1
Ключевые слова: голосовые технологии, вокальные биомаркеры, искусственный интеллект, голосовые боты, персонализированная медицина
Daria Hemmerling,
AGH University of Science and Technology, Faculty of Electrical Engineering, Automatics, Computer Science and Biomedical Engineering, Krakow, Poland
Benedetta Signorelli, Department of Human Health Science, Faculty of Medicine and Surgery, University of Florence, Italy Wojciech Wojakowski, Division of Cardiology and Structural Heart Diseases, Medical University of Silesia, Katowice, Poland
Michal Tendera,
Division of Cardiology and Structural Heart Diseases, Medical University of Silesia, Katowice, Poland
Tomasz Jadczyk
Division of Cardiology and Structural Heart Diseases, Medical University of Silesia, Katowice, Poland, International Clinical Research Center, St. Anne's University Hospital Brno, Brno, Czech Republic
VOICE TECHNOLOGY IN CARDIOVASCULAR DISEASES
Keywords: voice technology, vocal biomarkers, artificial intelligence, voice chatbots, personalised medicine
Daria Hemmerling,
Fan va Texnologiya Universiteti, Elektrotexnika, аvtomatika kompyuter fanlari va biotibbiyot muhandisligi fakulteti, Krakov, Polsha Benedetta Signorelli, Inson salomatligi fanlari bo'limi, Tibbiyot va jarrohlik fakulteti, Florensiya universiteti, Italiya Voytsex Vojakovski, Kardiologiya va yurakning strukturaviy kasalliklari bo'limi, Sileziya tibbiyot universiteti, Katovitse, Polsha
Mixal Tendera,
Kardiologiya va yurakning strukturaviy kasalliklari bo'limi, Sileziya tibbiyot universiteti,Katovitse, Polsha
Tomaç Jadchik
Kardiologiya va yurakning strukturaviy kasalliklari bo'limi, Sileziya tibbiyot universiteti,Katovitse, Polsha, Xalqaro klinik tadqiqotlar markazi, Brno Sent-Anna universiteti kasalxonasi, Brno, Chexiya
YURAK QON TOMIR KASALLIKLARIDA OVOZ TEXNOLOGIYASI
Kalit so'zlar: ovoz texnologiyasi, vokal biomarkerlar, sun'iy intellekt, ovozli chatbotlar, moslashtirilgan tibbiyot
Introduction
Despite great efforts, cardiovascular diseases (CVD) remain the leading cause of death worldwide [1]. Thus, novel diagnostic and treatment solutions are highly demanded to address current challenges in the field of clinical medicine. Interestingly, recent studies indicate a potential use of voice technology which covers a wide spectrum of artificial intelligence (AI) techniques allowing for human language understanding as well as for predictive analysis of vocal biomarkers. Physiologically, voice is the sound produced with the usage of the lungs and the vocal folds in the larynx. The vibration of vocal folds is generated when the air is pushed through vocal folds with sufficient pressure. On the one hand, the spoken language is the easiest and fastest way of communication. On the other hand, generation of voice requires using a series of coordinated, complex movements in the head, neck, chest and abdomen muscles, which impact the signals' frequency and amplitude resulting in specific, decodable sounds. By its complex nature, voice is an unique bio-print characteristic for each person conveying information about individual's personality, mood and health status. From a diagnostic point of view, voice is a bio-signal that can be acquired non-invasively and in an easy, economically-sound manner [ 1, 2]. Subsequently, a correlation between CVD and alterations in speech characteristics open new diagnostic opportunities based on deviations of voice features associated with CVD-mediated systemic inflammatory process which impacts anatomical structures responsible for voice generation [2]. Despite the very well-developed digital technology there is still the challenge to extract specific, important information about the patient's health condition. Especially due to its complex and dynamic characteristic, voice can be pronounced in different intonations and with different emotions. AI-driven digital solutions are still being sought on how to non-invasively evaluate patient's voice organs and effectively distinguish between patients with existing disorders and healthy individuals.
Moreover, the advancements in the field of computer science leveraged application of human-computer voice interfaces (also called voice assistants, voice chatbots or conversational agents) allowing machines to understand spoken language and generate human-like voice. The aforementioned implementation of voice technology in clinical field provides interesting tools which useability is currently being evaluated and tested [3].
This article covers the application of AI-based voice chatbots and the potential application of vocal biomarkers in the field of cardiovascular medicine.
Artificial intelligence-driven voice technology in medicine
Definition of voice assistant
Voice assistants (VA) powered with the advanced algorithm of AI and natural language processing allow for verbal communication between humans and computers. These conversational agents (i.e., Amazon Alexa, Apple's Siri or Google Assistant) can be installed on standalone devices called smart speakers or deployed on smartphones. The emulated human-machine conversations are based on the application of neural networks which perform voice-to-text analysis and text-to-voice computation generating natural human voice transforming day-to-day clinical practice [3, 4].
Application of voice technology in clinical practice
Voice-enabled technologies have the potential to influence everyday cardiovascular medicine practice by:
(1) Foreign language interpretation and real-time language translation,
(2) Patient education,
(3) Medication reminders and prescription refills,
(4) Continuity of care,
(5) Automated and paperless collection of medical data,
(6) Remote long-term monitoring,
(7) Diagnostic value of vocal biomarkers.
Foreign language interpretation
There is an incremental need to address language barriers for patients whose health care workers do not speak their primary language. Voice technology provides tools that facilitate communication in a safe and effective manner. Panayiotou et al. reviewed digital language translation solutions in health care settings. Among 15 iPad-compatible applications including 8 voice-to-voice and voice-to-text translation apps, 2 services (Assist and Talk to Me) were found to be clinically adequate for everyday conversations on subject matters that do not require a professional interpreter [5].
Patient education
There are numerous potential applications for the use of VA in the field of patient education and guidelines. Specifically, this Alexa-based applications can be used to provide information on the cardiopulmonary resuscitation (i.e., The Mayo Clinic First Aid) [6] or information from Mayo Clinic experts on topics related to cardiovascular diseases providing an access to the verified medical knowledge [7]. Furthermore, the Answers by Cigna application available on Amazon Alexa provides health coach programs supporting treatment plans. Furthermore, users can ask a wide range of health-
related questions receiving easy-to-understand responses [8]. A similar approach is exemplified by the Orbita ENGAGE designed for patients who can communicate verbally with a VA for medically associated frequently asked questions, especially based on symptom screening [9].
Medication reminders
Among CVD patients, medication non-adherence is a perceptible challenge both in the period immediately following an acute cardiovascular event as well as during long-term follow-up [10]. VA have been successfully implemented to support pharmacotherapy management. By saying to Alexa "Manage my medication" or "Refill my prescription", registered patients can set reminders to take medication and request prescriptions with home drug delivery through the Giant Eagle Pharmacy voice application [11].
Continuity of care
In the broader spectrum, some VA solutions like Orbit Connect are engineered for long-term follow-up as well as for pre- and post-visit through digital coaching, assessments, and care team communication. Furthermore, a holistic approach to patients with CVD should include mental status evaluation. Importantly, the prevalence of depression in this group is 3-fold higher in comparison to the general population [12]. Correspondingly, the Talk space voice application for Amazon Alexa allows users to access depression assessments tools as well as guided mindfulness techniques [13].
Automated and paperless collection of medical data
Integration of the medical voice AI chatbots with hospital electronic health systems (EHR) leverages advances in voice technology allowing for seamless and automatic population of electronic forms [14]. Noteworthy, it is crucial to ensure adequate level of security and privacy during transmission and computation of patient's protected health information (PHI). Accordingly, the GDPR (EU) and HIPAA (USA) regulations must be implemented for each software solution dealing with PHI.
Practical application of voice chatbot in clinical settings was exemplified by the CardioCube® service deployed on Amazon Echo smart speaker for automatic collection of patient-reported medical history at the Cardiology Outpatient Clinic of the Cedars-Sinai Medical Center (Los Angeles, CA, USA) [14]. Initialization of CardioCube® voice assistant was evoked by a verbal command "Computer, open CardioCube". Furthermore, patients answered pre-defined clinical questions which corresponded to the hospital intake form i.e. "Do you have high blood pressure?", "Have you ever had a heart attack?", "Have you been diagnosed with diabetes?". The answers provided verbally were translated into text using cloud-based AI systems and automatically populated a patient's record in the hospital EHR system. Healthcare providers could access the complete report through a standard web-based interface. This interactive approach was shown to streamline repetitive and time-consuming tasks during patient registration providing a secure and high accuracy (97.5%) digital tool automatically generating medical reports.
Remote long-term monitoring
The FCNcare by CardioCube® solution was implemented at the Family Care Network (Bellingham, WA, USA) for remote long-term follow-up of patients with diabetes and heart failure [15]. Individuals enrolled in the pilot study received Amazon Echo-deployed CardioCube® software for home use based on reporting actual clinical status during scheduled conversation sessions between patient and CardioCube®. The voice-based questionnaire consisted of eight questions: (1) "In the past week, have you missed any dose of your medication?", (2) "Are you needing a medication refill?", (3) Do you have any medication-related questions that you need your care team to answer?", (4) A caring reminder, eating more carbohydrates increases your blood sugar. All sugary foods contain carbohydrates, as do bread, rice, pasta, and potatoes. Have you been carefully managing your carbohydrate intake in the past week?", (5) "And how about exercises, how many times in the past week have you exercised?", (6) "As for this past week, were you able to check your sugar levels with a glucometer?", (7) "And how many times in the past week did you check
your blood sugar level?", (8) Were the majority of your readings in a good range?". Obtained results were analysed and automatically transferred to the Family Care Network EHR system for review by the nurse. Importantly, in case of health status deterioration (i.e. patient reports dyspnea) red-flagging notifications were implemented to improve useability of the service giving healthcare providers a quick access to the most crucial reports.
Diagnostic value of vocal biomarkers
In the literature, there are only a few studies that analysed the voice and speech signals in an acoustic parametrized manner for heart diseases. The researchers from Mayo Clinic reported a possible relationship between specific vocal biomarkers and coronary artery disease (CAD) underscoring the potential use of this simple biomarker to identify patients at risk [2]. The authors have analysed if patient voice signal characteristics are associated with the presence of CAD. They performed detailed acoustic analysis to describe the overall shape of signal's spectral envelope. With further analyses, authors identified five-voice features that were associated with CAD. Combining data with the Atherosclerotic Cardiovascular Disease risk scores, it was possible to identified two voice features that were independently associated with CAD (odds ratio OR = 0.37; 95% CI, interquartile range IQR = 0.180.79; and OR = 4.01; 95% CI, IQR = 1.25-12.84; p=.009 and p=.02, respectively). Both features were more strongly associated with CAD when patients were asked to describe an emotionally significant experience. The work was further developed and described by Maor et al. [16], where the authors have analysed if the vocal biomarker is associated with hospitalization and mortality among patients with congestive heart failure (CHF). By extracting a total of 223 acoustic features for each patient, the main novel finding of this study was that non-invasive voice signal characteristics are associated with adverse clinical outcomes among patients with symptomatic CHF [16]. Moreover, Pareek et al. [17] have also evaluated CAD patients. The results revealed significant variations in spectrograms and specific voice analyses between active and control group including jitter, shimmer, and complex parameters such as Relative Average Perturbation being as a quantitative measure of the voice.
Extraction of acoustic parameters enables an objective assessment of the voice and speech quality. The registration of the signal might be done in various manners. New technologies in digital signals processing enable the recordings without the requirement for access to an anechoic chamber. The sessions might be done at the doctor's office, at home, with a relatively low level of noise. Most smart speakers and VA-deployed on smartphones have a circular microphone array to provide voice-only interaction from a distance in standard room conditions. To perform the analysis with desired goals such as automatic diagnostic or highlighting health impairments using voice it must be stated what should be recorded. Voice signals might be recorded in different manners, depending on what features are desired. The phonation of sustained vowels with continuous phonation over a certain time are helpful to find discontinuities in signal's amplitudes and frequencies as well as changes in loudness levels. The speech recordings bring more information about the speech speed, pauses length, pitch and loudness changes. Accordingly, the speech might be acquired from a text read, a story-tell, a question-answer scenario, repetition of specific syllables and conglomerate of words. This enables the semantic voice analysis and extraction of meaningful words, enabling syntax analysis for natural language processing. Future directions
The aforementioned use cases confirm the feasibility of using voice chatbots and vocal biomarker application in the field of cardiovascular medicine. Noteworthy, VA can be integrated with the existing healthcare ecosystems leveraging clinical adoption opportunities of voice technology. The further development will enable constant patient monitoring with an immediate warning in case system detects health status deterioration including analysis of "invisible" vocal biomarkers. Such approach might be useful in predicting risk of the occurrence of health- and life-threatening conditions.
Список литературы/ References/ Iqtiboslar
1. Kones R, Rumana U. Cardiometabolic diseases of civilization: history and maturation of an evolving global threat. An update and call to action. Annals of Medicine. 2017;49(3):260-74. Epub 2016/12/13.
2. Maor E, Sara JD, Orbelo DM, Lerman LO, Levanon Y, Lerman A. Voice Signal Characteristics Are Independently Associated With Coronary Artery Disease. Mayo Clinic Proceedings. 2018;93(7):840-7. Epub 2018/04/17.
3. Hoy MB. Alexa, Siri, Cortana, and More: An Introduction to Voice Assistants. Medical Reference Services Quarterly. 2018;37(1):81-8. Epub 2018/01/13.
4. Suta P, Lan X, Wu B, Mongkolnam P, Chan J. An overview of machine learning in chatbots. International Journal of Mechanical Engineering and Robotics Research. 2020;9(4):502-10.
5. Panayiotou A, Gardner A, Williams S, Zucchi E, Mascitti-Meuter M, Goh AM, et al. Language Translation Apps in Health Care Settings: Expert Opinion. JMIR mHealth and uHealth. 2019;7(4):e11316. Epub 2019/04/10.
6. Mayo Clinic First Aid. Amazon. [21 NOV 2021]; Available from: https://www.amazon.com/mayo-clinic-first-aid/dp/b0744ljcv2.
7. Mayo Clinic News Network. Amazon. [21 NOV 2021]; Available from: https://www.amazon.com/mayo-clinic-news-network/dp/b0711hkbb6
8. Answers by Cigna. Amazon. [cited 21 NOV 2021]; Available from: https://www.amazon.com/Cigna-Answers-by/dp/B079SMKCZB
9. Employee Health Manager. Orbita. . [21 NOV 2021]; Available from: https://orbita.ai/orbita-employee-health-manager/
10. Simon ST, Kini V, Levy AE, Ho PM. Medication adherence in cardiovascular medicine. British Medical Journal. 2021;374:n1493. Epub 2021/08/13.
11. New ways to manage your medications at home using Alexa. [21 NOV 2021]; Available from: https://www.aboutamazon.com/news/devices/new-ways-to-manage-your-medications-at-home-using-alexa.
12. Chaddha A, Robinson EA, Kline-Rogers E, Alexandris-Souphis T, Rubenfire M. Mental Health and Cardiovascular Disease. The American Journal of Medicine. 2016;129(11):1145-8. Epub 2016/10/25.
13. Talkspace: Online therapy for all. Amazon. [21 NOV 2021]; Available from: https://www.amazon.com/Talkspace-Online-therapy-for-all/dp/B085VHRW33.
14. Jadczyk T, Kiwic O, Khandwalla RM, Grabowski K, Rudawski S, Magaczewski P, et al. Feasibility of a voice-enabled automated platform for medical data collection: CardioCube. International Journal of Medical Informatics. 2019;129:388-93. Epub 2019/08/25.
15. FCNcare by CardioCube. [21 NOV 2021]; Available from: https://jmir.org/api/download?alt_name=jmir_v23i5e22959_app1.mov&filename=62989d3b55a0fbfb33d8e0aac8c1a6bd.mov.
16. Maor E, Perry D, Mevorach D, Taiblum N, Luz Y, Mazin I, et al. Vocal Biomarker Is Associated With Hospitalization and Mortality Among Heart Failure Patients. Journal of the American Heart Association. 2020;9(7):e013359. Epub 2020/04/03.
17. Pareek V, Sharma R, editors. Coronary heart disease detection from voice analysis. 2016 IEEE Students' Conference on Electrical: Electronics and Computer Science (SCEECS); 2016.
