CC BY
4
UDC 81'33
Вестник СПбГУ Язык и литература. 2024. Т. 21. Вып. 1
Konstantin I. Belousov
St. Petersburg State University,
7-9, Universitetskaya nab., St. Petersburg, 199034, Russia
Perm State University,
15, ul. Bukireva, Perm, 614068, Russia
Aleksandar Taleski
Perm State University,
15, ul. Bukireva, Perm, 614068, Russia
Konstantin V. Ryabinin
St. Petersburg State University,
7-9, Universitetskaya nab., St. Petersburg, 199034, Russia
Perm State University,
15, ul. Bukireva, Perm, 614068, Russia
Natalia V. Boronnikova
Perm State University,
15, ul. Bukireva, Perm, 614068, Russia
The impact of communicative parameters on the speaker's spatial orientation in virtual reality*
For citation: Belousov K. I., Taleski A., Ryabinin K. V., Boronnikova N. V. The impact of communicative parameters on the speaker's spatial orientation in virtual reality. Vestnik of Saint Petersburg University. Language and Literature. 2024, 21 (1): 269-287. https://doi.org/10.21638/spbu09.2024.114
In this paper, we try to show a specificity of the speaker's spatial orientation system as a complex of cognitive functions in virtual reality (VR). Spatial orientation can be egocentric, al-locentric or geocentric depending on which verbal deictic occurs as a point of reference. The use of one or another type of spatial orientation in VR can be conditioned by various factors. We explore the impact of communicative parameters on the speaker's spatial orientation in VR. For that purpose, a VR experiment was conducted by applying five scenes with different communicative parameters. The consideration of the connection between the communicative parameters embedded in five different scenes and cognitive processes in the speaker's orientation in VR is a novelty. The specific features of speaker orientation are presented in terms of three different deictic and communicative aspects: the opposition person-oriented and distance-oriented systems; the communicative situations themselves and their internal pa-
* The research was carried out with the support of St. Petersburg State University, project code 94034584.
© St. Petersburg State University, 2024
rameters; the location of the referent in proximal, medial and distal positions. 24 informants participated in the experiment, and 725 reactions presented in the form of lines were obtained as material which was analyzed and processed in the Semograph Information System. The results showed that in the communication in VR, the allocentric orientation prevails even with the proximal position of the referent, the deictic coordinate system can be fully integrated into some communicative situations, and certain communicative parameters can affect the speaker's spatial orientation. We discuss the mechanisms that might explain the results and offer recommendations for future experiments.
Keywords: spatial orientation, virtual reality, point of references, communication, deixis. Introduction
In this paper, we try to show the impact of communicative parameters on the speaker's spatial orientation (SO) in the virtual reality (VR) environment and specific features of the deictic system in VR.
VR is a reality created by certain technologies and technical means that provide partial or complete immersion of a person in this display and create the illusion of reality, and the ability to interact with a three-dimensional (3D) visual or other sensory environment. The creation of VR is based on the use of computer technologies that make it possible to fully realize a person's immersion in an artificial world, which is perceived through sensory stimuli (such as images and sounds) and interactive devices that send and receive information and are worn in the form of glasses, headphones, gloves or suits [Bohil et al. 2009; Lowood 2021; Peeters 2019].
VR as an environment modeled using a computer technology, was initially researched within the technical sciences in connection with the development of information technologies and later was covered by the humanities. VR as a new field of research appeared in the 90s of the last century in some humanities such as psychology, sociology, linguistics, etc. VR is defined as an experimental environment intended for researching, treatment, training and also used as a research method [Biocca, Levy 1995; Blascovich et al. 2002; Pfeiffer 2012; Pan, Hamilton 2018; Peeters 2019].
As an experimental environment, VR has a number of advantages over traditional experiments. Thus, traditional experiments, where stimuli are included one by one on a computer monitor, are replaced in VR experiments by immersing participants into VR itself. Participants become a part of naturalistic 3D scenes. In addition, VR has the ability to generate real and made-up situations, where they reflect different language modalities (speech, gestures, gaze, facial expressions) and can communicate with objects, people and other elements that are limited only by the human imagination, and experimenters can manipulate virtual objects and environment parameters [Fox et al. 2009; Peeters 2019: 898].
Easy and fast modeling and adaptation of the VR environment and its parameters satisfy the needs of the goals and tasks of the experimenters makes VR suitable for research and analysis of SO, which is described as a set of cognitive functions or as an ability to maintain a sense of location in different and new environments for humans [Buckely et al. 2016; Gramann 2013; Pastel et al. 2020; Wilson, Soranzo 2015; Wolbers, Hegarty 2010].
The complex architecture of human spatial cognition is reflected in the speaker's behavior in VR in terms of SO, coordination, and navigation, all of which have been the subject of extensive research in the last three decades [Becker et al. 2019; Byagowi, Moussavi
2012; Creem-Regehr et al. 2015; Gramann 2013; Interrante et al. 2006; Jones et al. 2019; Keshner, Kenyon 2009; Leyrer et al. 2011; Mohler et al. 2006; Morganti et al. 2013; Pastel et al. 2020; Phillips et al. 2012; Wartenberg et al. 1998]. Differences exist between our research and these inquiries across multiple dimensions. Within these literature pieces, the focus points of SO within VR encompass a wide array: novel technological modalities, intricate technical variables, emerging neuroscientific methodologies, cognitive modalities, and more. Several among these studies adopt a comparative approach, juxtaposing VR-based SO against its real-world counterpart. Furthermore, a predominant trend among these scholarly works is the amalgamation of spatial coordination and navigational paradigms within the rubric of SO.
In our research which emerges as a result of an experiment in a VR environment, we define the spatial reference frames and orientation based on the communicative situation and its elements. Spatial coordination and navigation in VR are not included in our research, and SO means orientation from a certain point of reference in the communication process. Based on responses derived from diverse simulated communicative VR scenes, there is potential tobuild models of the speaker's communicative and deictic behavior and to determine the speaker's coordinate and orientation system in VR.
It should be noted that various interpretations of theorientation system exist. The orientation system reviewed in the researches [Gramann 2013; Klatzky 1998; Pastel et al. 2020; Shelton, McNamara 1997] is defined binarily as two distinct spatial reference frames. The first one is the egocentric reference system which specifies the location and orientation relative to the navigator's body. The second one is the environmental (allo-centric) reference system which uses the positional relations of surrounding recognizable objects outside the navigator.
The theory of the ternary frame-of-reference system model developed by S. Levinson is the most appropriate one for our research. Levinson's orientation system is based on specifying angles and directions to determine the location of the object (referent) relative to a certain point of reference. Levinson identifies three types of orientation systems in space: built-in (intrinsic), relative, and absolute. The systems differ in their internal structure [Levinson 1996; 2003].
The built-in orientation system is based on a dual spatial relationship (referent — point of reference), e. g. the ball (referent) is to the left of you (point of reference). In this coordinate system, there is a coincidence of a point of reference (any object in space) and the beginning of the system. This model is also known as an allocentric indication (non-egocentric object-centered), where the objects are localized regardless of the position of the speaker (observer) [Levinson 1996; 2003].
The absolute system is also a double spatial relation based on an already fixed coordinate axis, which does not depend on the shape of the point of reference. This system is also known as geocentric, and it is built on the basis of a landscape or an abstract scheme that refers to some natural point of reference such as light sides, landscape elements, etc., e. g. The house is uphill [Bryant 1992; Levinson 1996; 2003]. Unlike the papers [Gramann 2013; Klatzky 1998; Pastel et al. 2020; Shelton, McNamara 1997], where the geocentric orientation is part of the allocentric orientation, we see it as a separate concept conditioned only by the objects of the environment as landmarks. In our research, the allocentric orientation is the addressee orientation. We believe that although the addressee is an avatar and is not fully interactive (except for some body and head movements), its role in the
communicative situation and the task performed by the speaker hints of its uniqueness in the perception of the speaker in relation to other objects in the VR environment.
For a relative coordinate system, an observation point is important to determine the position of the referent in relation to the point of reference. Here, the observation point and the beginning of the system coincide, where the observation point and the beginning of the system is the speaker's position, e. g. John is here. This system is based on the so-called anthropocentric or egocentric approach. It is often perceived as deictic and egocentric, but its secondary forms may not always relate to the speaker [Bryant 1992; Lev-inson 1996; 2003]. For the relative system, the anthropocentric approach is the main one. Nevertheless, language SO can be conducted not exclusively with relation to the speaker (observer), but also in relation to other participants.
In the relative system, the observer determines his position in space from the point of view of the connection between a human body and its parts with the environment (physiological perception of reality) [Berthele 2006; Fillmore 1975]. The coordinates of the human body or parts of other physical objects (face, head, leg, horn, root, etc.) are used in both allocentric and geocentric coordinate systems [Levinson 1994]. Thus, in the physiological space, this is based on the proprioceptive perception of reality, the speaker, when oriented, proceeds not only from parts of his own body, but also from the body of other participants in the communicative act [Anokhin 1968].
When determining the position of objects relative to their own body, the speaker presents himself as a part of a certain space, which is finite and has a certain structure. Thus, the speaker shows how this object contacts him or other objects in this area. The point of reference is a speaker or any other randomly selected object. If the location of the speaker-observer acts as a reference point, without which it is difficult to understand the essence of the statement, then we are talking about the deictic use of language elements. Otherwise, when the speaker's location is secondary at the moment of speech, deictic elements are used non-deictically [Fillmore 1975: 235-237]. According to Yu. Apresyan, the utterance is understood non-deictically with an allocentric or geocentric orientation, when there is no observer, and only the referent object and the reference object are present [Apresyan 1995: 278]. As for egocentric orientation, it is assumed that the observer is mentally introduced by the speaker into the number of direct participants in the described situation. The figures of the speaker and the observer may or may not coincide.
In addition to the speaker-observer, the communicative dyad — the speaker and the addressee — acts as the basis of the deictic center. A locative act implies the presence of a speaker and an addressee, who are the main participants in a dialogic speech. B. Uspenskiy defines deixis as a phenomenon of a dialogic speech where the speaker and the addressee change their positions in the process of communication. According to him, the space of the addressee can be combined with the space of the speaker, and can also be opposed to him. This suggests that with spatial deixis, the orientation to the addressee is carried out [Uspenskiy 2011: 8]. Ch. Fillmore also suggested that the point of reference (deictic center) of the physical organization of space could be the addressee. According to Fillmore, "spatial deixis is that aspect of deixis which involves referring to the locations in space of the communication act participant; it is that part of spatial semantics which takes the bodies of the communication act participants as significant reference objects for spatial specification" [Fillmore 1982: 37].
Orientation strategies and spatial reference frames as a set of cognitive functions are flexible and can fluctuate in a certain communicative situation both in reality and in VR. In VR, various factors such as intrinsic individual components, technical means and technologies, including display performance and environmental characteristics can determine the choice of speaker in one or the other orientation strategies and spatial reference frames. Our paper examines the differences in choosing the orientation strategies that may vary and are highly relevant to the orientation performance, depending on some communicative parameters.
SO as an objective characteristic of cognition, finds numerous representations in language, expressed in the primary meanings of specialized words to denote space — spatial deictics. We think that communicative and deictic parameters could affect the type of orientation and coordination of the speaker in the communication process in VR. The ways of SO of the speaker in virtual reality, characteristic of different types of scenes and types of communicative behavior are determined by analyzing spatial and personal deictics. Here the term "deictic" includes only lexical (pronouns, adverbs) means and syntagmas with indicative semantics.
The main goal of this paper is to present circumstances in terms of communicative parameters. The specific features of speaker orientation are considered from three different deictic and communicative aspects. The first is from the aspect of the opposition person-oriented and distance-oriented situations. For a distance-oriented system, the distance between the deictic center and the referent is important. The person-oriented system is characterized by the correlation of the deictic with the participants of communication [Fillmore 1982; Anderson, Keenan 1985: 282-286; Jungbluth 2003; Diessel 1999: 39]. The second is from the aspect of the communicative situations themselves and their internal parameters, and the third is from the position of the referent in three different positions (proximal, medial and distal).
The necessity of this research arises from the importance of acquiring linguistic information regarding cognitive mechanisms within the examined VR, including virtual spatial orientation. Induced by the overarching communicative context or by the specific communicative variables, the cognitive processes of the speaker within a virtual setting remain insufficiently researched. These processes consider the communication-specific feature in VR, questioning the deictic behavior of a person and including the SO of the participants of a VR communicative act.
Methodology
Impetus and participants
We conducted an experiment in a VR environment. In the experiment, we used five scenes (three — person-oriented, two — distance-oriented). We took schemes for creating experimental scenes in VR from the work of A. Rostovtsev-Popel' [Rostovtsev-Popel' 2009].
All five scenes (see Fig. 1) help us model different situations of interaction between communication participants — two communicants (a speaker marked as S and a VR addressee marked as A) and a referent (a bottle). Each of the scenes is based on three communicative parameters: the relative position of the speaker and the addressee in the com-
Fig. 1. Schémas of the five scenes used in the experiment
municative act; the location of the referent in the internal/external communication space; the visibility of the referent in the field of the communicants' vision or outside it. The field of vision of communicants in Fig. 1 is shown by the direction of the arrows.
If we look deeper into the scenes, we can underline that the first scene (Sc1) is person oriented and includes the following parameters: face-to-face communication, where the referent is located in the internal space of communication, and in the field of the communicants' vision. The second scene (Sc2) is person oriented and represents face-to-face communication. The referent is located in the external communication space. It is in the field of the addressee's vision, but behind the speaker's back. In the process of communication, the speaker turns to look at the referent and the referent automatically turns to the left/right of him or in front of him. The third scene (Sc3) is also person oriented and it represents face-to-face communication. The referent is placed in the external communication space, in the field of the speaker's vision, behind the addressee. The fourth scene
Fig. 2. Fragment of the experiment
(Sc4) is distance oriented and has two varieties. In the first case, the speaker is next to the addressee. The second case considers the informant's activity (moving) in a VR environment, and the speaker often finds himself behind the addressee. We will consider both varieties. The referent is located in the external communication space, in the field of vision of both communicators. The fifth scene (Sc5) is distance oriented. The speaker stands to the right of the addressee. The referent is located in the external communication space, in the field of the speaker's vision and out of the field of the addressee's view, behind his back. When the speaker refers to the addressee, he sometimes turns his body or head in the addressee direction, and, thus, the referent is to the left of the speaker.
In addition to these parameters, an important aspect of the scenes is the appearance of the referent in three different positions in each scene. The first position where the referent appears is proximal (marked as R1 in Fig. 1). It indicates that the object is located close to the speaker. The second position of the referent is medial, i. e. the referent is in a neutral position in relation to the communicants (marked as R2 in Fig. 1). The third position of the referent is distal (marked as R3 in Fig. 1), which refers to a remote object relative to the speaker.
In the experiment, 24 Russian-speaking participants took part. In order to collect the data, the participants' task was to ask the addressee (an avatar) to take what the speaker sees on the table (a referent). The whole process was video recorded. The videos were recorded using two devices: an internal camera filming the scene the informant was looking at in a VR helmet, and an external camera filming the gestures. The synchronization of the two videos allowed us to unite the verbal with the non-verbal behavior of the informants (see Fig. 2). That ratio of verbal and non-verbal behavior is represented in the form of reaction at the stage of material processing as follows:
Ex. 1. Мхм. Рядом с вами на столе стоит бутылка, возьмите ее [делает движение
вперед ладонями, сжатыми в замок]. "Mhm. There is a bottle on the table next to you, take
it [makes a movement forward with palms clenched into the lock]."1
The total number of line reactions received is 725.
Processing of experimental data
The material was processed in the Semograph Information System2. At the first stage, we created a project which allocated a separate context with a relevant data description for each reaction of the subjects. Each reaction was described by a set of metatext and social parameters. At the next stage of data processing, we analyzed each (non)verbal expression. The analysis of the expressions determines the deictic or communicative semantics of each phrase.
Depending on the semantics, each syntagma belongs to one or another deictic and communicative class. A single syntagma can be included in several classes independently or in combination with other (non)verbal tools, based on its properties and functions. Thus, we provided classification categories and created a universal classifier consisting of three central categories based on certain deictic and communicative parameters.
The semantic criterion used to distinguish classifier units is based on the general meaning of a certain class of words. It involves attributing a specific word to a broad conceptual category which is characterized by a generalized meaning. Each category in our universal classifier is a hierarchical semantic structure that consists of subcategories, classes, subclasses, groups, subgroups, and components.
Semograph IS can automatically calculate the volumes of the received classes, as well as the frequency of the sharing of categories in the studied syntagmas. Thus, the Semograph IS provided us with quantitative data on verbal elements with spatial and personal semantics. At the next stage, we calculated the frequency of occurrence as the ratio of the absolute frequency fi to the number of lines. The number of lines varies, and we can use all lines to calculate the overall proportion of occurrence or to calculate the proportion of occurrence for a specific communication sample in a specific amount.
Results
Type of Orientation
In this paper, firstly, we have referred to the personal subclass based on the personal deictics. The personal subclass defines the participants of communication. It determines the appearance of two types of orientation: the subclass "I" indicates egocentric orientation, and the subclass "You" indicates allocentric orientation. Secondly, the subclasses in the category "Space" show the geocentric orientation based on the spatial deictics. Table 1 shows the proportion of occurrence in all three types of orientations.
1 The speaker's movements are described in parentheses. The structure of the example is as follows: first the original in Russian is shown (important verbal elements are bolded), then transliteration in italics, and finally an English translation, separated by quotation marks.
2 Available at: https://semograph.org (accessed: 29.02.2024).
Table 1. The overall proportion of occurrence in three types of orientation in VR
Type of orientation Overall proportion of occurrence
Egocentric 0.462
Allocentric 1.716
Geocentric 0.877
Based on the frequency of occurrence described in Table 1, it can be emphasized that the speaker has an allocentric orientation most often (1.716), i. e. relative to the addressee (a VR character). The speaker in these situations uses the forms of personal pronoun— You (Ex. 2).
Ex. 2. Она по-прежнему позади вас, только теперь она находится справа от вас. Заберите, пожалуйста, бутылку. "It is still behind you, only now it is on your right. Please take the bottle."
The second most common type of orientation in VR communication is geocentric (0.877). Here, the speaker, pointing to the referent, orients himself in space with the help of virtual objects, which we designate as a topos (table, chair) and a border (wall, bar counter). This type of orientation is shown in the following example:
Ex. 3. Я надеюсь [сцепляет руки перед собой], вам не составит труда взять красную бутылку, которая стоит на светлом квадратном столике, которая стоит [замешательство, поднимает кулак левой руки] около барной стойки на том конце зала."1 hope [clasps your hands in front of you] you won't have any trouble picking up a red bottle that stands on a light square tablethat stands [confused, raises a fist of left hand] near the bar counter at the other end of the hall."
The third type of orientation that we have encountered in the reactions of speakers is egocentric. Egocentric orientation is determined by the use of the deictic I and its case variants, when the speaker is oriented relative to himself (Ex. 4). The proportion of occurrence of egocentric deictics in the lines of informants is less than that of allocentric and geocentric.
Ex. 4. Аа... Мне кажется, я могу сама взять эту бутылочку [указывает левой рукой на референт открытой ладонью, поворачивает голову к экспериментатору], она близко как-то ко мне [смех]. Ну, дай, пожалуйста, [приподнимает левую руку] бутылочку, которая находится прямо передо мной. "Ah... I think I can take this bottle myself, [points with his left hand to the referent with an open palm, turns his head towards the experimenter] it is somehow close to me [laughter]. Well, give me, please, [raises his left hand] the bottle that is right in front of me."
Type of Deictic Orientation Systems Parameter
The first communicative aspect, which helps us look at the variability of the SO of the speaker in VR communication, is the type of deictic orientation (person-oriented and distance-oriented systems) shown in the Table 2.
Table 2. Proportion of occurrence in three types of orientation in VR in person-oriented and
distance-oriented scenes
Type of orientation Person-oriented Distance-oriented
Egocentric 1.044 0.413
Allocentric 4.169 1.764
Geocentric 2.002 0.941
As can be seen from Table 2, the speaker is allocentrically oriented in both person-oriented and distance-oriented scenes. In person-oriented scenes, allocentric orientation is two times more common than geocentric and four times more common than egocentric. We have an identical situation in the distance-oriented scenes, where the allocentric orientation is almost twice and four times higher than the geocentric and egocentric. It can also be seen that the proportion of occurrence in the person-oriented situations is slightly more than twice as high as that of the distance-oriented situations in all three types of orientations.
Internal Communicative Parameters of Five Scenes
To better understand the reason for such data, it is necessary to look a little deeper into situations. If we look separately at the situations, then we can observe uniformity in the orientation of the speaker in VR. The variability of the speaker orientation in each of the five scenes is shown in Table 3.
Table 3. Proportion of occurrence in three types of orientation in VR in five scenes
Type of orientation Scenes
Sc1 Sc2 Sc3 Sc4 Sc5
Egocentric 0.549 0.730 0.257 0.368 0.458
Allocentric 1.771 1.215 2.111 1.764 1.764
Geocentric 0.722 0.945 0.868 0.903 0.979
From Table 3, we can see that most often in all five scenes, the subjects used allocentric orientation. Geocentric and egocentric orientations are used less. In Sc1, the speaker's allocentric orientation was two and a half times more common than geocentric and almost three times more common than egocentric. As for the Sc2, the allocentric orientation was realized in it not much more relative to the other two. In Sc3, in contrast to Sc1 and Sc2, informants were oriented allocentrically three times more often than geocentrically, and seven and a half times more often than egocentrically. Situations Sc4 and Sc5 are identical, since they had an allocentric orientation almost two and a half times more geocentric and almost five times more egocentric.
Table 4 shows the proportion of occurrence in varieties of geocentric orientation in VR in five scenes. We can underline that the orientation on topos prevails over the orientation on the border. It is also noted that the overall proportion of occurrence in a Sc1 differs from the sum of the topos and the border.
Table 4. Proportion of occurrence in varieties of geocentric orientation in VR in five scenes
Varieties of geocentric orientation Scenes
Sc1 Sc2 Sc3 Sc4 Sc5
Topos 0,771 0,680 0,736 0,743 0,764
Border 0,021 0,194 0,132 0,160 0,215
The Location of the Referent Parameter
The orientation of the speaker in the VR communication can be determined by the location of the referent. The Table 5 illustrates the proportion of occurrence of three types of speaker orientations in the VR communication in terms of the position of the referent in three different points (proximal, medial and distal).
Table 5. Proportion of occurrence in three types of orientation in VR depending on the referent's position
Type of orientation R1 R2 R3
Egocentric 0.638 0.392 0.362
Allocentric 1.663 1.7 1.813
Geocentric 0.617 0.883 0.733
The speaker at the proximal (R1), medial (R2) and distal (R3) position of the referent is primarily oriented allocentrically. The situation with egocentric and geocentric orientation is different. Thus, in the proximal position of the referent, the egocentric and geocentric orientations are represented almost identically, with a slight advantage of the egocentric. In the medial and distal position of the referent, the speaker in communication in VR is more often geocentrically oriented than egocentrically. The proportion of occurrence is more than twice as high in the geocentric orientation as in the egocentric one.
If we look horizontally in Table 5, we can see that the egocentric orientation is higher for a little less than twice when the referent is in the proximal position as opposed to the medial and distal ones. Allocentric and egocentric orientation are almost identical with insignificant deviations in all three positions of the referent.
Discussion
Based on the results of the use of spatial and personal deictics by informants, we determined the specific features of the speaker's SO in VR. The speaker, first of all, focuses allocentrically.
The reason for the high advantage of the allocentric orientation over the other two, in our opinion, can be sought in the task itself, which the informants were obliged to fulfill. Namely, the task involves the speaker to ask the addressee to give him the specific referent. Hence, for a more efficient fulfillment of the task, it is natural for the speaker to be oriented from the aspect of the addressee, because the addressee is the one who should fulfill the task.
The orientation on the addressee is nothing new in terms of SO. Many linguists (e. g.: [Anokhin 1968; Uspenskiy 2011; Fillmore 1982]) have suggested the existence and infrequent application of this type of orientation.
The allocentric orientation of the speaker reveals certain varieties. The current study found that the speaker can focus both on the addressee's body as a whole (Ex. 2) and on its parts, primarily the arms and back as in the following example:
Ex. 5. Передайте, пожалуйста, бутылочку, которая находится за вами, по правую руку от вас. "Please pass on the bottle behind you, on your right hand."
This confirms the theory of some linguists [Anokhin 1968; Levinson 1994] set out in background that certain parts of the addressee's body can also be used as point of reference. G. Opalka also pointed out that often we orient ourselves not only with the help of the speaker's body, but also with the help of the addressee, in which case utterance and orientation are not egocentric, but duocentric [Opalka 1982].
Another important finding was that the speaker adds different spatial coordinates to the forms of personal deictics. Thus, in certain situations informants can determine the distance by using units — meter, step, etc., as in the following examples:
Ex. 6. Бутылочка [взмах левой рукой в сторону бутылки] находится, которую вас прошу передать, перед вами [ритмично взмахнула кистью левой руки] на расстоянии примерно трех метров на прямоугольном столе с краю. "Bottle [wave your left hand to the side of the bottle] is on the, that you are asked to pass on, before you [rhythmically waved his left hand] at a distance of about three meters on a rectangular table's edge." Ex. 7. Возьмите бутылку, находящуюся на 13 часов от вас, в паре шагов. "Take a bottle that is at 13 o'clock away from you, a couple of steps away."
Using expressions which can determine the distance by using units like in Ex. 6, we can define it as a tendency to impose norms and habits from reality on VR. In Ex. 7, we are talking about a spatial pointing characteristic in the field of aviation and medicine [Mariner 2007]. The speaker imagines a dial in front of him, where the position of the arrow at 12 o'clock indicates the location of the object at the top or in front, at 3 o'clock — on the right, at 6 o'clock — at the bottom or behind, at 9 o'clock — on the left. The remaining sections indicate an intermediate position. In this example, there is a certain deviation when using such an expression. The speaker used the form of the pronoun You which guided him relative to the VR character but not relative to himself. The reaction was obtained in the Sc1, where the referent is in the medial sphere in relation to the communicants.
The next type of orientation, which the speaker in VR communication relies on, is the geocentric one that can be seen from Table 1. The geocentric orientation is often conditioned by the objects, which are located next to the referent, more precisely in the sphere of the referent. Thus, the speaker is oriented primarily with the help of chairs, tables (topos), and then in relation to the spatial borders of the VR-room.
As for the geocentric type of orientation, it should be noted that often when pointing to the referent in VR, the speaker focuses not only on the object itself, but also on the parts of this object, e. g. ...перед вами на расстоянии примерно трех метров на прямоугольном столе с краю. "...in front of you at a distance of about three meters on a rectangular table's edge." In this situation, the speaker, using an expression table's edge,
pointed to a certain part of this virtual object. This suggests that the speaker defines different spaces within the VR space.
Regarding the varieties of geocentric orientation — topos and borders (Table 4), we can say that the use of topos as a point of reference is noted more often than the use of borders. In addition, it should be emphasized that in the distance-oriented situation Sc5, the second geocentric variety — the border — is used twice as often as in Sc2, Sc3, and Sc4. Here the referent is in the external space of communication, behind the VR character, out of his field of view, but in the field of the speaker's view. The presence of the referent outside the field of the character's view determines the use of the border as a point of reference not only when determining the location of the referent, but also for establishing space.
In the person-oriented situation Sc1, geocentric orientation to the border is insignificantly detected. This is explained by the fact that in this situation the referent is located in the internal space of communication, between locators (see Fig. 1). In this case, the border is marked with communicants, more precisely, the "border" as a point of reference loses its relevance due to the positions of communicants in space.
One unanticipated finding was that the third frequency orientation, which we find in the reactions of the informants, is the egocentric one (see Table 1). Although it was expected that the speaker in the orientation in the field of VR, above all, will apply the habits acquired in reality, there are still deviations in this regard. This phenomenon is observed in the use of egocentric orientation. This is so if it is known that the egocentric-ity of space means that many spatial characteristics are determined by the position of the subject (consciousness, appearance, speech), this quality of space acts as the main one in the linguistic reflection of reality.
In order to take a deeper look at the prevalence of egocentric orientation of the speaker in VR communication, we can refer to the results in Table 5. As can be seen, the results in Table 5 are identical to the results in Table 1 in terms of what allocentric orientation prevails over others, and geocentric is often second in use by the speaker. But here it can be noticed that for the first time the egocentric orientation is found in slightly more reactions than the geocentric one (in R1). This is not surprising given that this data is observed in situations where the referent is in a proximal sphere relative to the speaker. The most interesting finding in relation to the data from Table 5 was that the egocentric orientation of the speaker even in the proximal position of the speaker is in the second position behind the allocentric and slightly ahead of the geocentric (0.021).
This means that in the situation when the referent is in a proximal position in relation to him, the speaker does not orient by putting himself as a point of reference, and most often the point of reference in the orientation is the VR addressee.
A possible explanation for this might be that the speaker does not feel the VR space. It is based on the previous studies which noted that the microcosm of a person is in the center of the linguistic worldview [Gak 1998: 127], which determines the presence in the spatial concept of two main spheres, connected and unrelated to the speaking subject. From this theory, we can assume that the speaker divides the VR space into two polar parts — opposition I-sphere, where he is oriented egocentrically and where it is logical to put himself as a point of reference, and non I-sphere, where he is oriented allocentrically and geocentrically. The I-sphere and non I-sphere reflect the opposition "my own and not my own." Thus the I-sphere is something of its own and known, and not something foreign and unknown, as in the case of the VR space.
Another possible explanation for this is that the speaker uses communicative strategies depending on the needs of the environment. Thus, using the VR addressee as the center of orientation, the speaker thinks that in that way he will influence more efficient and successful execution of the communicative act. On the other hand, geocentric orientation can be considered as auxiliary, and egocentric as a template of reality, which is inserted in virtual reality as a routine in situations where the referent is in a proximal position in relation to the speaker.
Our findings revealed one more important fact. Namely, in Table 3 the proportion of occurrences in egocentric orientation is nearly two or more times higher in Sc2 than in the Sc3, Sc4 and Sc5.
If we take into account that Sc2 is person-oriented like Sc3 and the only difference is the variation of one communicative parameter into two, i. e. a parallel existing of the referent in the field of communicants' vision and outside of it. From here, that additional parameter is likely to be identified.
Another assumption is that the order of the scenes themselves can have a certain effect on the high proportion of occurrences. If the referent is between the communicators and in their field of vision, making it easier to determine the location of the referent in Sc1, then in this situation the speaker does not see the referent in face-to-face communication with the addressee.
The degree of integration of the spatial reference frames and orientation systems can also be an explanation. Sc2 is very thoroughly integrated into the deictic coordinate system, involving mostly egocentric and less the other two. Also, an important aspect which these results point to is that VR proposes communicative situations which need more deictics in order to successfully implement a communicative act. Thus, more effort is required from the speaker to orient the addressee, i. e. to give him an adequate coordinate system. In Sc2, the reverse process is observed. More precisely, unlike other situations, here the speaker uses fewer deictics to complete a successful communicative act.
Conclusion
The purpose of the current study was to determine specific features of the deictic system and SO of the speaker in the VR communication. Many factors can influence the choice of the deictic system and the type of orientation in VR. It points to the second aim of this study which was to investigate circumstances in terms of communicative aspects (person-oriented and distance-oriented situations and their communicative parameters, the appearance of the referent in three different positions) that influenced the choice of the type of SO.
This study has shown that the speaker in VR communication is oriented allocentri-cally (to the VR addressee). The geocentric orientation was the second most common orientation that we encountered in the reactions of the informants, and the egocentric one occupies the last place in the choice of the speaker, i. e. the speaker is the least oriented, putting himself in the role of a starting point of reference. Such an image is characteristic of almost all communicative indicators, with the exception of the location of the referent. Thus, the egocentric orientation is not the last choice in the proximal position of the referent in relation to the speaker and slightly exceeds the geocentric one.
The second major finding was that the egocentric deictic coordinate system could be thoroughly integrated into certain communicative situations. This can be seen in the Sc2 which includes orientations relative to the speaker and whose ratio of occurrence of deictics is nearly two or more times higher than the proportion of occurrence of deictics used in the subsequent 3 scenes.
Collectively, these findings indicate that the communicative parameters concerning the referent's location (proximal, medial, or distal) and its presence within or beyond the communicants' visual field, upon which the communicative context is constructed, have the potential to influence the speaker's selection of Spatial Orientation (SO) and its distinct attributes.
This is the first study of substantial duration which examines associations between the communicative parameters implemented in five different scenes and the cognitive processes in the speaker orientation in the field of VR.
The generalizability of these results is subject to certain limitations. For instance, we believe that the task set in this way could affect the results. In addition, parts of the speaker's body (arms, legs, etc.) are absent in his field of vision in the VR environment. Also, the changing schedule of the scenes could affect the final result.
The study should be repeated using another task and other technical means (a suit and gloves) which will enable the speaker to see his hands, feet, etc. during the communicative acts. Our hypothesis is that in this case, the proportion of occurrence of the allocentric orientation should decrease due to an increase in favor of the other two. We do not exclude the possibility that the total number of verbal actions will decrease due to the use of gestures by the speaker.
References
Anderson, Keenan 1985 — Anderson S. R., Keenan E. Deixis. In: Language Typology and Syntactic Fieldwork.
Vol. 3. Shopen T. (ed.). Cambridge: Cambridge University Press, 1985. P. 259-308. Anokhin 1968 — Anokhin P. Biology and neurophysiology of the conditioned reflex. Moscow: Meditsina Publ., 1968. (In Russian)
Apresyan 1995 — Apresyan Yu. Selected Works. Lexical semantics. Synonymous language means. 2nd ed., rev.
and add. Vol. I. Moscow: Shkola "Iazyki russkoi kul'tury" Publ., 1995. (In Russian) Becker et al. 2019 — Becker J., Meyer U., Eichler T., Draheim S. A Supernatural VR Environment for Spatial User Rotation. In: IEEE Conference on Virtual Reality and 3D User Interfaces (VR). Abbott D. (ed.). 2019. P. 850-851. https://doi.org/10.1109/VR. 2019.8798290 Berthele 2006 — Berthele R. Ort und Weg. Berlin: Walter de Gruyter, 2006. https://doi.org/10.1515/97831-10890464
Biocca, Levy 1995 — Biocca F., Levy M. R. Communication applications of virtual reality. In: Communication in the age of virtual reality. Jennings B., Dolf Z. (eds). Hillsdale: Erlbaum. 1995. P. 127-157. https://doi. org/10.4324/9781410603128 Blascovich et al. 2002 — Blascovich J., Loomis J., Beall A. C., Swinth K. R., Hoyt C. L., Bailenson J. N. Immersive Virtual Environment Technology as a Methodological Tool for Social Psychology. Psychological Inquiry. 2002, 13 (2): 103-124. https://doi.org/10.1207/S15327965PLI1302_01 Bohil et al. 2009 — Bohil C., Owen C. B., Jeong E., Alicea B., Biocca F. Virtual reality and presence. In: Twenty first century communication: a reference handbook. Eadie W. F. (ed.). Thousand Oaks: SAGE Publications, Inc. 2009, P. 534-544. https://doi.org/10.4135/9781412964005 Bryant 1992 — Bryant D. J. A spatial representation system in humans.Psycholoquy. 1992, 3 (16): 1. Buckely et al. 2016 — Buckley J., Phelan J., Seery N., Canty D. Assessing visual perception in virtual reality environments. In: Tiernan P. (ed.). Proceedings of the 33rd International Manufacturing Conference. Limerick: University of Limerick, 2016.
Byagowi, Moussavi 2012 — Byagowi A., Moussavi Z. Design of a Virtual Reality Navigational (VRN) experiment for assessment of egocentric spatial cognition. In: 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, San Diego, California, USA. San Diego, 2012. P. 4812-4815. https://doi.org/10.1109/EMBC. 2012.6347070 Creem-Regehr et al. 2015 — Creem-Regehr S. H., Stefanucci J. K., Thompson W. B. Chapter six — Perceiving absolute scale in virtual environments: How theory and application have mutually informed the role of body-based perception. Psychology of Learning and Motivation. 2015, (62): 195-224. https://doi. org/10.1016/bs.plm.2014.09.006 Diessel 1999 — Diessel H. Demonstratives: Form, function, and grammaticalization. Philadelphia: John
Benjamins Publishing Company, 1999. https://doi.org/10.1075/tsl.42 Fillmore 1975 — Fillmore C J. Lectures on Deixis 1971. In: Indiana University Linguistics Club. Berkeley:
University of California, 1975. P. 217-306. Fillmore 1982 — Fillmore C. J. Towards a Descriptive Framework for Spatial Deixis. In: Space, Place and Action: Studies in Deixis and Ralated Topics. Jarvella R., Klein W. (eds). Chichester: Wiley, 1982. P. 3159.
Fox et al. 2009 — Fox J., Arena D., Bailenson J. N. Virtual Reality: A survival guide for the social scientist.
Journal of Media Psychology. 2009, 21 (3): 95-113. https://doi.org/10.1027-/1864-1105.2L3.95 Gak 1998 — Gak V. Language transformations. Moscow: Shkola "Iazyki russkoi kul'tury" Publ., 1998. (In Russian)
Gramann 2013 — Gramann K. Embodiment of Spatial Reference Frames and Individual Differences in Reference Frame Proclivity. Spatial Cognition & Computation: An Interdisciplinary Journal. 2013, 13 (1): 1-25. https://doi.org/10.1080/13875868.2011.589038 Interrante et al. 2006 — Interrante V., Ries B., Anderson L. Distance perception in immersive virtual environments, revisited. Virtual Reality Conference, IEEE. 2006: 3-10. https://doi.org/10.1109/ VR.2006.52
Jones et al. 2019 — Jones J. A., Hopper J. E., Bolas M. T., Krum D. M. Orientation Perception in Real and Virtual Environments. IEEE Transactions on Visualization and Computer Graphics. 2019, 25 (5): 20502060. https://doi.org/10.1109/TVCG.2019.2898798 Jungbluth 2003 — Jungbluth K. Deictics in the Conversation Dyad: Findings in Spanish and Some Cross-linguistic Outlines. In: Deictic Conceptualization of Space, Time and Person. Lenz F. (ed.). Amsterdam; Philadelphia: John Benjamin Publishing Company, 2003. P. 13-40. https://doi.org/10.1075/ pbns.112.04jun
Keshner, Kenyon 2009 — Keshner E. A., Kenyon R. V. Postural and spatial orientation driven by virtual reality. Studies in Health Technology and Informatics. 2009, (145): 209-228. https://doi.org/-10.3233/978-1-60750-018-6-209
Klatzky 1998 — Klatzky R. L. Allocentric and egocentric spatial representations: Definitions, distinctions, and interconnections. In: Lecture Notes in Artificial Intelligence. Freksa C., Habel C., Wender K. (eds). Berlin; Heidelberg: Springer, 1998. P. 1-18. https://doi.org/10.1007/3-540-69342-4_1 Levinson 1994 — Levinson S. C. Vision, shape, and linguistic description: Tzeltal body-part terminology and object description. Linguistics. 1994, 32 (4/5): 791-856. https://doi.org/10.1515/ling.1994.32.4-5.791
Levinson 1996 — Levinson S. C. Frames of reference and Molyneux's question: Crosslinguistic evidence. In:
Language and Space. 1996: 109-169. https://doi.org/10.1146/annurev-.anthro.25.L353 Levinson 2003 — Levinson S. C. Space in language and cognition: Explorations in cognitive diversity.
Cambridge: Cambridge University Press, 2003. https://doi.org/10.1017/CB09780511613609 Leyrer et al. 2011 — Leyrer M., Linkenauger S. A., Bulthoff H. H., Kloos U., Mohler B. The Influence of Eye Height and Avatars on Egocentric Distance Estimates in Immersive Virtual Environments. In: Proceedings of the ACM SIGGRAPH Symposium on Applied Perception in Graphics and Visualization (APGV '11), Toulouse, France. Toulouse, 2011. P. 67-74. https://doi.org/10.-1145/2077451.2077464 Lowood 2021 — Lowood H. E. (n. d.). Virtual Reality. Encyclopedia Britannica. Retrieved May 13, 2021,
from https://www.britannica.com/technology/virtual-reality (accessed: 15.03.2023). Mariner 2007 — Mariner L. Cleared for Takeoff: English for Pilots. Book 1. California: AE Link Publications, 2007.
Möhler et al. 2006 — Möhler B. J., Creem-Regehr S. H., Thompson W. B. The influence of feedback on egocentric distance judgments in real and virtual environments. In: Proceedings of the 3rd symposium on Applied perception in graphics and visualization (APGV '06), Boston Massachusetts USA. Boston, 2006. P. 9-14. https://doi.org/10.1145/1140491.1140493 Morganti et al. 2013 — Morganti F., Stefanini S., Riva G. From allo- to egocentric spatial ability in early Alzheimer's disease: A study with virtual reality spatial tasks. Cognitive Neuroscience. 2013, 4 (3-4): 171-180. https://doi.org/10.1080/17588928.2013.854762 Opalka 1982 — Opalka H. Representations of Local Ni-Deixis in Swahili in Relation to Bühler's "Origo des Zeigfelds". In: Here and There. Weissenbord J., Klein W. (eds). Amsterdam: John Benjamins, 1982, P. 65-79.
Pan, Hamilton 2018 — Pan X., Hamilton A. F. C. Why and how to use virtual reality to study human social interaction: The challenges of exploring a new research landscape. British Journal of Psychology. 2018, 109 (3): 395-417. https://doi.org/10.1111/bjop.12290 Pastel et al. 2020 — Pastel S., Chen C. H., Bürger D., Naujoks M., Martin L. F., Petri K., Witte K. Spatial orientation in virtual environment compared to real-world. Journal of Motor Behavior. 2020, 53 (6): 693-706. https://doi.org/10.1080/00222895.2020.1843390 Peeters 2019 — Peeters D. Virtual reality: A game-changing method for the language sciences. Psychonomic
Bulletin & Review. 2019, 26 (3): 894-900. https://doi.org/10.3758/s13423-019-01571-3 Pfeiffer 2012 — Pfeiffer T. Using virtual reality technology in linguistic research. In: Virtual Reality Workshops (VR), IEEE, Costa Mesa, CA, USA. Costa Mesa, 2012. P. 83-84. https://doi.org/10.1109/-vr.2012.6180893
Phillips et al. 2012 — Phillips L., Interrante V., Kaeding M., Ries B., Anderson L. Correlations Between Physiological Response, Gait, Personality, and Presence in Immersive Virtual Environments. Presence. 2012, 21 (2): 119-141. https://doi.org/10.1162/PRES_a_00100 Rostovtsev-Popel' 2009 — Rostovtsev-Popel' A. A. Typology of demonstrations: Average deictics.
Voprosyiazykoznaniia. 2009, (2): 22-34. (In Russian) Shelton, McNamara 1997 — Shelton A. L., McNamara T. P. Multiple views of spatial memory. In: Psychonomic
Bulletin & Review. 1997, 4 (1): 102-106. https://doi.org/10.3758/BF03210780 Uspenskiy 2011 — Uspenskiy B. Deixis and secondary semiosis in language. Voprosyiazykoznaniia. 2011, (2): 3-30. (In Russian)
Wartenberg et al. 1998 — Wartenberg F., May M., Preruch P. Spatial orientation in virtual environments: Background considerations and experiments. In: Spatial cognition. Freksa C., Habel C., Wender K. (eds). New Jercy: Lawrence Elrbaum Associates, Inc., 1998. P. 469-489. https://doi.org/10.1007/3-540-69342-4_22
Wilson, Soranzo 2015 — Wilson C. J., Soranzo A. The Use of Virtual Reality in Psychology: A Case Study in Visual Perception. In: Computational and Mathematical Methods in Medicine. 2015, ID 151702. https://doi.org/10.1155/2015/151702 Wolbers, Hegarty 2010 — Wolbers T., Hegarty M. What determines our navigational abilities? Trends in Cognitive Sciences. 2010, 14 (3): 138-146. https://doi.org/10.1016/j.tics.-2010.01.001
Received: March 15, 2023 Accepted: November 3, 2023
Белоусов Константин Игоревич
Санкт-Петербургский государственный университет,
Россия, 199034, Санкт-Петербург, Университетская наб., 7-9
Пермский государственный национальный исследовательский университет,
Россия, 614068, Пермь, ул. Букирева, 15
Талески Александар
Пермский государственный национальный исследовательский университет,
Россия, 614068, Пермь, ул. Букирева, 15
Рябинин Константин Валентинович
Санкт-Петербургский государственный университет,
Россия, 199034, Санкт-Петербург, Университетская наб., 7-9
Пермский государственный национальный исследовательский университет,
Россия, 614068, Пермь, ул. Букирева, 15
kostya.ryabinin@gm ail.com
Боронникова Наталия Владимировна
Пермский государственный национальный исследовательский университет,
Россия, 614068, Пермь, ул. Букирева, 15
Влияние коммуникативных параметров на пространственную ориентацию говорящего в виртуальной реальности*
Для цитирования: Belousov K. I., Taleski A., Ryabinin K. V., Boronnikova N. V. The impact of communicative parameters on the speaker's spatial orientation in virtual reality. Вестник Санкт-Петербургского университета. Язык и литература. 2024, 21 (1): 269-287. https://doi.org/10.21638/spbu09.2024.114
В работе делается попытка показать специфику системы пространственной ориентации говорящего как комплекса когнитивных функций в виртуальной реальности (VR). Пространственная ориентация может быть эгоцентрической, аллоцентрической или геоцентрической в зависимости от того, какой дейктик используется в качестве точки отсчета при ориентации в виртуальной реальности. Использование того или иного типа пространственной ориентации в виртуальной реальности может быть обусловлено различными факторами. В работе исследуется, как коммуникативные параметры, встроенные в смоделированные VR-сцены, могут повлиять на пространственную ориентацию говорящего. С этой целью был проведен эксперимент в VR-среде с использованием пяти сцен с различными коммуникативными параметрами. Рассмотрение связи между коммуникативными параметрами, внедренными в пять различных сцен, и когнитивными процессами ориентации говорящего в виртуальной реальности предлагается впервые. Особенности ориентации говорящего представлены в рамках трех различных дейктических и коммуникативных аспектов: оппозиция личностно-ориен-тированных и дистанционно-ориентированных систем; сами коммуникативные ситуации и их внутренние параметры; положение референта в проксимальной, медиальной и дистальной позициях. В эксперименте приняли участие 24 информанта. В качестве материала было получено 725 реакций, представленных в виде размеченных реплик, которые были проанализированы и обработаны в информационной системе «Семо-
* Работа выполнена при поддержке СПбГУ шифр проекта 94034584.
граф». Результаты анализа показали, что при общении в виртуальной реальности ал-лоцентрическая ориентация преобладает даже при проксимальном положении референта, дейктическая система координат может быть полностью интегрирована в определенные коммуникативные ситуации, а определенные коммуникативные параметры могут влиять на пространственную ориентацию говорящего. В статье обсуждаются механизмы пространственной ориентации, которые могли бы объяснить результаты, и предлагаются рекомендации для будущих экспериментов.
Ключевые слова: пространственная ориентация, виртуальная реальность, точка отсчета, коммуникация, дейксис.
Статья поступила в редакцию 15 марта 2023 г.
Статья рекомендована к печати 3 ноября 2023 г.
