CC BY
29
THE MULTI-DOMAIN INFOCOMMUNICATION MODEL AS THE BASIS OF AN AUDITORY INTERFACES DEVELOPMENT FOR MULTIMEDIA INFORMATIONAL SYSTEMS
Alexander D. Sotnikov,
The Bonch-Bruevich St.Petersburg State University of Telecommunications, St.Petersburg, Russia, adsotnikov@mail.ru
Gleb G. Rogozinsky,
The Bonch-Bruevich St.Petersburg State University of Telecommunications, St.Petersburg, Russia, gleb.rogozinsky@gmail.com
Keywords: auditory interfaces, modified domain model, human machine interfaces, sonification.
Most of the up-to-date informational systems experience the problem of a limitation of user ability to percept the significant informational streams. The multimedia visual and auditory human machine interfaces (HMI) can be applied in the multiple technological systems, though created mostly in artistic way and by artists, without systematized approach to their design and formalization of the demands.
Purpose. The creation of effective human machine interfaces demands development of corresponding description methods, analysis and construction of the object characteristics and processes in the technological systems in a form of acoustic signals with given demands for adequacy and accuracy of data representation. Methodology. As the fundamental base for description and analysis of the problem we used the domain model of infocommunication systems, which provides high-level formalization of the processes of information interaction, served by corresponding user interfaces. Findings. The Modified Multi-domain Model is proposed for the generalized description of the meaningful characteristics of informational systems. The model provides formalization for the interface design based on the sonification methods.
Information about authors:
Alexander D. Sotnikov, Department of IT in Economics, Professor, Sc.D., The Bonch-Bruevich St.Petersburg State University of Telecommunications, St.Petersburg, Russia
Gleb G. Rogozinsky, Deputy Head of Medialabs, Ph.D., The Bonch-Bruevich St.Petersburg State University of Telecommunications, St.Petersburg, Russia
Для цитирования:
Сотников А.Д., Рогозинский Г.Г. Мультидоменная модель инфокоммуникаций как основа построения аудиальных интерфейсов для мультимедийных информационных систем // T-Comm: Телекоммуникации и транспорт. 2017. Том 11. №5. С. 77-82.
For citation:
Sotnikov A.D., Rogozinsky G.G. (2017). The multi"domain infocommunication model as the basis of an auditory interfaces development for multimedia informational systems. T-Comm, vol. 11, no.5, рр. 77-82.
In the modern world of high technologies the interaction between human user and the different technological systems becomes more and more complex and diversified. Such interaction produces the intense informational load (and its saturation stale known as informational overload) oil the user and causes several types of malfunctions in the control circuit. Such situations stipulate the usage of the multimedia interfaces, able to split informational load between different sensory systems of a human user. Thus the overall informational load will be demultiplexed into several parallel informational streams, received by the corresponding sensor of a human user. The list of applications may theoretically include everything between a smart oven with a simple audio alann system to analysis unit of the multiparametric complex system and interpretation of its concrete characteristics in various tasks of acoustic tomography or seismic data analysis. All those methods are known under the common name Bonification or combination of methods of representing any data with the non-speech sounds. Herewith, the utilization of non-speech acoustical signals allows decreasing the reaction time of the user by utilization of the biological peculiarities of primary sensory areas of human brain.
I. Key concepts and definitions
The key categories and concepts for infocommunications are those of "information", "information interaction", "information process", "information object". As a fundamental scientific term, information lacks a concise and universally accepted definition 111. This lack of unity means that the category is insufficiently developed and can be defined and interpreted in new ways.
Information interaction. The category of interaction reflects the mutual impacts of various objects, their mutual causality and change of slate. The information process, as a realization of a specific information interaction, is defined by a number of authors as "... non-energetic interaction of objects, characterized by transfer {generation and internalization) of ideal categories (senses, meanings, images, emotions)" or "... interaction of objects leading to the change in the knowledge possessed by at least one of them" [1 J.
The domain model assumes dissection of the interaction field into three interacting layers (domains), each of which is linked to the group of the objects of a common nature - Physical Domain (PD), Informational Domain (ID) and Cognitive Domain (CD). The control of technical objects and systems assumes formation of solutions for control interactions, which alter the state of the system towards increasing of target function according to information representation of the object.
The PD is typically concerned with the energy processes and the interaction of material world objects. The situation analysis and intellectual activity producing evaluations and solutions is the product of mental and psychic activity of the CD. The ID is the area for the circulation of data used in the CD, representing the objects, phenomena and process of the PD.
At the domain borders the corresponding interfaces perform the information interaction between different elements of the system. Each object/subject of the system can be characterized by the finite number of states, represented by it own thesaurus.
Thus, the object A of a PD with corresponding thesaurus
states, mapped onto the multiplicity of informational representa-
£
tions (AfA of the thesaurus
f.
(1)
In other words the information is transferred, when the signal transmitting an image (notion) from the varifold thesaurus of the source system A into the varifold thesaurus of the target system B is changed.
The information is received when a new image of the source is formed within the varifold thesaurus of the target system.
(2)
where Q^Qj ~ mapping operators between different domains, i.e. PD, ID or CD; thesauri of a target signal and a source
signal.
In detail, we can write (2) as following
rf
(3)
Thus the mapping between the corresponding domains is actually the operation of information impact between entities of the domains, expressed in the discovery of maximum conformity between elements of thesauri
Information impact is the influence of the "source" A on the state of the "target" B, which manifests itself in the change of the image (B), seen in the variety of elements within the thesaurus of
the "source" (B )v4. Since the "source" 4a and "target" ¿.b thesauri are different, the original image in the internal thesaurus
of {£?}-" and the image of {By** in the "source" thesaurus are also different. This can lead to errors, or inaccurate representation of the object in the thesaurus of the user.
Information exchange is the receiving and transmission of
s
signals leading to the mutual alteration of images {A)^B and
{ B)■"'' of the exchange participants. This can be caused by alteration (expansion) of the participants* thesauri and
Information interaction is the mutual change of images of
s s
own systems oi (/i)1*'1 and ( B) leading to the change ot images
(A)^B and {B)$A in the other participants.
Information system (IS) is a system containing "information" and providing i( to the user. A necessary condition is as follows: "The necessary components of an IS are: the user and the potential information". A sufficient condition is "The user and the potential information form an IS", IS are made up of elements which are information images (A) of the real (material and immaterial) entities A and possess information significance.
Information significance is the property of representing the entity, which requires a descriptive method containing a set of basic "meanings", immanent to the entity. "Information significance" should not be confused with "value" or "usefulness"; these properties are related to the users and the possibility of satisfying their need of information.
The formalized set of "meanings" is the 4a thesaurus. The entity item has a number of discernible slates, which are perceived by the observer as a set of images of an object, each having its own "meaning". The number of states determines the potential information carried by the object. When the observer acquires an image of the object (by means of perception and recognition), the
potential information is actualized on the basis of the information representation of the object. The potential presence of information in system A is determined by the sei of discernible states of the system and the varifold system thesaurus
Perception of the information transferred occurs when the receiver R acquires a new image of system A in the varifold receiver thesaurus
User U is a person, object or process capable of perceiving
images and possessing its own receiver thesaurus
Potential information is the set of discernible states of the entity S in the set of representations (S)'based on the thesaurus 4s> possessing information significance.
Actual information is the representation (image) in
the thesaurus I\v of the user U.
Information System S:
is the combination comprised of system entities, users, the system thesaurus and user thesauri, the set of information representations of the system (potential information), and the set of system images (representations in user thesauri).
Communication System C.'
{C, (CA Q&&) is the combination of consecutive transformations
niiic Qii'8
"V
ensuring the representation of the set of source images ( A ) in
the set of target images ( B ) via the set of states (images) {C) of signal C given required precision.
Telecommunication System T:
is the combination of consecutive transformations
signals and their representations, regardless of the spatial location of the source A and target B. (AzD, B€D,D*AnB).
obtained and adequately presented, which has important repercussions.
1. Fintities possessing potential information are "inside" the ICS, and their informational representations are placed there as well. The inclusion of an object into a system (Fig. 1) presupposes the coincidence between the subset of the object thesaurus with the set of the system thesaurus , resulting in the coincidence between representations
A : object
S: information system Fig, I. Information "inside" the system
2. Entities are "outside'1 the system (Fig. 2). In this case, the system and object thesauri are linked by the correlation
Zse&h and only an approximation of images {A)^ak{A)^s is possible. It is determined by the proximity of the thesauri i^, c^, i.e. the information is incorrectly represented by the system.
Infocommunication System F:
{S, & < 5u, &« 5 C, 4c, (C)&; is a combination including the entities of the information system S and telecommunication system T.
II. ICS Classification
Within the framework provided by these definitions, it is possible to classify ICS according to a number of criteria which characterize different aspects of the interaction between the system, the user and the information, important from the point of view of the analysis and subsequent design of ICS [2, 3]. In this context, the spatial distance between the users is of no consequence, and the notions of ICS and IS are the same thing.
Information placement. Information objects (representations of entities) are an integral ICS element. The entities themselves (carriers of potential information) are not always elements of the ICS.
The presence or absence of entities within the ICS is the first classification criterion, which determines the way in which entities and/or information objects (i.e. potential information) are
Fig. 2. Information "outside" the system
The relationship between the user and IS. This is the second criterion for ICS classification. The user can be either "inside the system" or "outside the system". The terms "inside" and "outside" reflect the interaction possibilities between the system and use thesauri, rather than spatial or topological characteristics.
1. The user is "within the system" (Fig. 3), when:
a) The system and user thesauri are entirely identical
This is a case of confluence, when the user is identified with the system and can no longer be regarded as the "receiver" of the information, because he possesses the entirety of images arising from the varifold system thesaurus, while the IS cannot serve as a source of information, the definition is violated and the usefulness of the IS is lost.
b)The user thesaurus can be arbitrarily changed (expanded) by means of adding elements of the system thesaurus, which is initiated by the user, whose thesaurus can then be "reconciled" with the system thesaurus tjf^u to a degree.
e)The system thesaurus can be arbitrarily changed by the user.
U: user
S: information system Fig. 3. User inside the system
In cases when "the user is inside the system", there is no operation of mutual transformation of thesaurus elements Instead, one subset is represented in the other F: £—>£/, i.e. it is possible to perceive the image adequately. The degree of adequacy is determined by the degree of thesaurus coincidence, i.e. the degree of proximity between sets sr.
2. The user is outside the system (Fig. 4), when: a) The user ijo system thesauri are autonomous and cannot be reconciled to an arbitrary degree L'at the user's initiative", but only "at the request of the system", as a result of providing information to the user.
A : object U : user
S : information system
Fig. 5. User and data are "outside" the ICS
Let the entity, the IS and the user with their thesauri and representations accordingly be:
{A. 4bM-P), {5, &,..($)%, {U, %v,(U)S%
then the image of the entity created in the user thesaurus corresponds to the two-way exchange between the entity, the IS and the user, defined by the sequence
W-e^f/ff^^f^Kff®
where the expression in brackets describes the "reproduction" of
image in the image created in the varifold the-
saurus of the user.
The general view of an elementary interaction between two information systems within the ID is a unidirectional transfer/receiving of the representation of the object ofPD, described as follows
,4)
Where A„ - entities of cognitive (CD) or physical domains (PD); - thesaurus of entity A; Q - mapping of entity A„ on
the user thesaurus C^.
The information interaction occurs by means of exchanging "messages", which are the subsets of the set of information representations {A of the object A„, transferred between informational systems Cm and C^.
In more common case the process of the informational interaction in the system which consists of several informational sources (in other words, the entities of CD and PD) and several targets is described as
Fig. 4. User outside the system
When the user is "outside the system", the thesaurus transformation operation becomes significant where
S'eS. or more rarely which corresponds to the change
(expansion) of the thesaurus, provides for the expansion of entity space and for the greater precision in the perception of the image
of the object (A in the thesaurus of the user (</(>&)&. The correlation between the user, object and system thesauri (£[/, ih, 4s) is crucial for the classification of IS, since the image is created in the varifold thesaurus, i.e. the image is derived from
the thesaurus A • The most complicated case is when
the user and the entities are "outside" the system (Fig. 5). This produces the two-way exchange, which is described in the following way in the formal notation.
f.
<4 f-1
L Jlt«U
(5)
where N, M. K - the number of entities, which correspond to a physical (N), informational (M) and cognitive (K) domains.
The information process is a combination of elementary information interactions within the ID. Expression (5) represents the domain model, a description of information interaction linked to the nature of three corresponding domains, i.e. PD, ID and CD.
Let {A\, ... /In}€ A: the set of objects in the PD, {C/}e C: the set of objects in the ID, while the subsets of representations {C„,} are within m autonomous informational systems active within the ID. Thus, for objects A„ and consumers Bt generally:
(6)
Expression (6) represents the model of information interaction of two subjects of CD, expressed in terms of ID system representations and PD objects. The representation of object Aj in the user thesaurus B, serves as the basis for making decision
IR, by subject (B,) of CD.
(({A^yZiR^,
(7)
where —^ is a decision making operator.
The whole sequence of forming the image of an object At in the ID, its perception by the subject Bi in the CD, decision making and transfer via the ID to the subject B4 in the CD, can be
described as follows:
This expression represents the information interaction model for CD subjects in terms of ID system representations, PD objects and CD subject decisions.
Ill Application for auditory interfaces design
The formula (5) describes in general the process of informational interaction between physical objects aggregate and the entities of CD through the transfer of their informational representations. The Fig. 6 shows the simplified graphical interpretation of (5) for the case of auditoiy interface, as a part of HMI.
The informational representations of the PD objects (i.e. elements of technical systems) can be both visual objects (static or moving) and acoustical objects (stationary or changing). The latter are one of the research objects. The list of such objects can be expanded through the usage of olfactory (smell) and/or tactile (muscle) spaces. Such research leads us towards so-called multimodal interfaces study and corresponding design tasks [4].
Thus, at the process of development of 1IMI the following problems to be stated:
!. The problem of a set definition for the characteristics of the object, which includes the analysis of values ranges, temporal characteristics, the data rates. In general, the objects of a PD can be characterized by the following groups of descriptors:
- Parameters and characteristics which define the ei gen states of the objects;
- Parameters which define the space localization of the objects, their translation, velocity, acceleration and any other possible parameters, related to the coordinate alteration;
- Parameters which define the object-related set of events.
Generalizing, we can state that the main generators or sources
of the informational streams, demanding on their interpretation in the interfaces, are objects, events and processes, each of which possesses the specific set of the parameters and characteristics.
2. The problem of a definition of the parameters and characteristics of a human auditory system, related to the perception of acoustical signals. This problem seems to be more or less well solved, assuming considerable research results in the different fields of acoustic studies and hearing physiology. Meanwhile,
several problems related to the psychological aspects of hearing still cannot be defined as completely solved. It can be proved by considering the whole history of musical art development and such multidisciplinary subjects as a sound design [5].
3. The problem of an adequate and accurate mapping of the multiplicity of the entities of a PD thesaurus onto the multiplicity of ID thesaurus (or the parameters of a real presented objects onto parameters of acoustic signals) for the subsequent mapping onto the CD (thesaurus of cognitive domain).
We should mark here that the latter problem cannot be effectively solved by just formal correlating of elements of the two thesauri, since such approach does not account psycho emotional aspects of the sound perception, and thus it requires specific multidisciplinary approaches and development of some non-trivial criteria of possible solutions estimation.
The developed sound objects can be remarkably applied not only to the field of a system monitoring and a technical system and object control as the real entities of a PD, but also in such promising fields as augmented reality and virtual reality. The separate interesting field of the research application is the entertainment sector, which includes computer gaming, interactive art, and numerous art directions related to sound design applications and soundscapes.
Conclusion
1. The application of non-speech auditory interfaces in the complex human machine systems solves several actual tasks, related to the problems of stability increasing and effectiveness of such systems.
2. The modified multidomain model of infooommunication interaction provides general methodology which includes the methods of a formal description of informational processes, taking place into the systems, and the ways of quantitative analysis of the parameters and characteristics of utilized signaling systems. At the same time the model provides the possibility for accounting weakly formalized components of the interfaces, related to psycho physiological and emotional aspects of perception.
References
1. Polonnikov R.l. (2006). Key Concepts of the General Information Theory. St.Petersburg: NAUKA, 203 p.
2. Sotnikov A.D. (2004). Principles of the Applied Area Analysis in Healthcare lnfocommunieation Systems. Proceedings of Higher Educational Establishments in Communications, no. 171, pp. 174-183.
3. Sotnikov A.D. (2003). Classification and Models of Applied ln-Tocom mimical ion Systems, Proceedings of Higher Educational Establishments in Communications, no. 169, pp. 149-162,
4. Basov O.. Ronzhin A„ Budkov V„ Saitov t, (2015) Method of Defining Multimodal Information falsity for Smart Telecommunication Systems. In Balandin S., Andreev S„ Koucheryavy Y. (Eds,), Internet of Things, Smart Spaces, and Next Generation Networks and Systems. ruSMART 2015. Lecture Notes in Computer Science, vol, 9247. Springer, Heidelberg, pp. 163-173.
5. Chemy E., Lilius J., Brusila J., Mouromisev D.. Rogozinsky G. (2016) An Approach for Structuring Sound Sample Libraries Using Ontology. In Ngonga Ngomo AC., Kfemen P. (Eds.), Knowledge Engineering and Semantic Web. KESW 2016. Communications in Computer and Information Science, vol 649. Springer, Heidelberg, pp. 202-214,
ИНФОРМАТИКА
МУЛЬТИДОМЕННАЯ МОДЕЛЬ ИНФОКОММУНИКАЦИЙ КАК ОСНОВА ПОСТРОЕНИЯ АУДИАЛЬНЫХ ИНТЕРФЕЙСОВ ДЛЯ МУЛЬТИМЕДИЙНЫХ ИНФОРМАЦИОННЫХ СИСТЕМ
Сотников Александр Дмитриевич,
Санкт-Петербургский государственный университет телекоммуникаций им. проф. М.А. Бонч-Бруевича,
Санкт-Петербург, Россия, adsotnikov@mail.ru
Рогозинский Глеб Гендрихович1'2
НОЦ "Медиацентр", 2Санкт-Петербургский государственный университет телекоммуникаций им. проф. М.А. Бонч-Бруевича, Санкт-Петербург, Россия, gleb.rogozinsky@gmail.com
Дннотация
Большинство современных информационных систем сталкиваются с проблемой ограниченных возможностей восприятия пользователем значительных информационных потоков. Мультимедийные визуально-акустические и речевые человеко-машинные интерфейсы получают широкое применение в разнообразных технических системах, но создаются "творческим" путем, не имея систематизированного способа их описания и формулировки требований. Постановка задачи. Проектирование эффективных интерфейсов взаимодействия в человеко-машинных системах требует разработки методов описания, анализа и конструирования представлений свойств и характеристик объектов и процессов в технических системах в форме акустических пространств с заданными требованиями к адекватности и точности представления.
Метод исследования. В качестве фундаментальной основы для описания и анализа проблемы используется доменная модель инфокоммуникаций, позволяющая формально описать процессы информационного взаимодействия, обслуживаемые соответствующими интерфейсами участников процесса.
Результаты. Предложена модифицированная доменная модель инфокоммуникаций, позволяющая с единых позиций описывать значимые характеристики информационной системы, формулировать требования к интерфейсу и конструируемой акустической среде на основе известных технологий сонификации.
Ключевые слова: аудиальный интерфейс, модифицированная доменная модель, человеко-машинный интерфейс, сонификация. Литература
1. Полонников Р.И. Основные концепции общей теории иинформации. СПб.: Наука, 2006. 203 с.
2. Сотников А.Д. Принципы анализа прикладной области в инфокоммуникационных системах здравоохранения. Труды учебных заведений связи, 2004, №171. С. 174-183.
3. Сотников А.Д. Классификация и модели прикладных инфокоммуникационных систем, Труды учебных заведений связи, 2003, №169. С. 149-162.
4. Basov O., Ronzhin A., Budkov V., Saitov I. Method of Defining Multimodal Information Falsity for Smart Telecommunication Systems. In Balandin S., Andreev S., Koucheryavy Y. (Eds.), Internet of Things, Smart Spaces, and Next Generation Networks and Systems. ruSMART 2015. Lecture Notes in Computer Science, vol. 9247. Springer, Heidelberg, 2015, pp. 163-173.
5. Cherny E., Lilius J., Brusila J., Mouromtsev D., Rogozinsky G. An Approach for Structuring Sound Sample Libraries Using Ontology. In Ngonga Ngomo AC., Kiemen P. (Eds.), Knowledge Engineering and Semantic Web. KESW 2016. Communications in Computer and Information Science, vol 649. Springer, Heidelberg, 2016, pp. 202-214.
Информация об авторах:
Сотников Александр Дмитриевич, кафедра информационных технологий в экономике, профессор, д.т.н., Санкт-Петербургский государственный университет телекоммуникаций им. проф. М.А. Бонч-Бруевича, Санкт-Петербург, Россия
Рогозинский Глеб Гендрихович, НОЦ "Медиацентр", заместитель начальника, к.т.н.,Санкт-Петербургский государственный университет телекоммуникаций им. проф. М.А. Бонч-Бруевича, Санкт-Петербург, Россия
т
