CC BY
35J—J J""J J^) J J/J Laushkina А.А., Roslyakova S.V., Smirnov A.V. Implementation of adaptive lighting systems
Q'po'i t-ttt >-p *>-p to reduce stressful situations in multi-user spaces // Научный результат.
I Г.. ) У I I П I / А I Информационные технологии. - Т.5, №4,2020
UDC 004.9 DOI: 10.18413/2518-1092-2020-5-4-0-9
Laushkina A.A. Roslyakova S.V. Smirnov A.V.
IMPLEMENTATION OF ADAPTIVE LIGHTING SYSTEMS TO REDUCE STRESSFUL SITUATIONS IN MULTI-USER SPACES
Saint Petersburg National Research University of Information Technologies, Mechanics and Optics, 49 Kronverkskiy prospekt, St. Petersburg, 197101, Russia
e-mail: nastasjalausckina@mail.ru, svetlana.roslyakova@itmo.ru, sav_smirnof@mail.ru
Abstract
The analysis of scientific papers revealed that the majority of studies in the field of lighting control systems are devoted to energy conservation, and single-user interaction with lighting. At the same time, the issue related to the impact of lighting on human stress levels in an educational environment is covered only by a number of studies that have investigated lighting parameters for different types of human activities. While there are known dependencies of the positive influence of lighting on reducing conflict behavior and stress levels in multi-user spaces, there is no mechanism for adapting appropriate lighting systems to the psycho-emotional state of users of such spaces. The paper presents the analysis of scientific publications and patents on non-visual effects of lighting on human body. The effects of lighting on conflict behavior and stressful state of a person were studied when using adaptive lighting systems. The methods and approaches of the developed adaptive lighting systems and the constraints associated with these methods were analyzed. The results of theoretical and experimental studies are given. A set of methods and tools to create an intelligent multi-user adaptive lighting system is proposed.
Keywords: lighting; adaptive lighting systems; multi-user lighting systems; interactive lighting.
УДК 004.9
Лаушкина A.A. Рослякова С.В. Смирнов А.В.
РЕАЛИЗАЦИЯ АДАПТИВНЫХ СИСТЕМ ОСВЕЩЕНИЯ, ОБЕСПЕЧИВАЮЩИХ УМЕНЬШЕНИЕ СТРЕССОВЫХ СИТУАЦИЙ В МНОГОПОЛЬЗОВАТЕЛЬСКИХ ПРОСТРАНСТВАХ
Федеральное государственное автономное образовательное учреждение высшего образования «Санкт-Петербургский национальный исследовательский университет информационных технологий, механики и оптики», Кронверкский пр., д. 49, г. Санкт-Петербург, 197101, Россия
e-mail: nastasjalausckina@mail.ru, svetlana.roslyakova@itmo.ru, sav_smirnof@mail.ru Аннотация
В ходе анализа научных работ выявлено, что большинство исследований в области систем управления освещением посвящено вопросам энергосбережения, и однопользовательскому взаимодействию с освещением. При этом вопрос, связанный с воздействием освещения на уровень стресса человека в образовательной среде, освещает лишь ряд исследований, изучавших параметры освещения для различных видов деятельности людей. При известных зависимостях позитивного влияния освещения на снижение конфликтного поведения и уменьшение стрессовых ситуаций в многопользовательских пространствах отсутствует механизм адаптации соответствующих систем освещения к психоэмоциональному состоянию пользователей таких помещений. В связи с этим, был проведен анализ научных публикаций и патентов по теме невизуального воздействия освещения на организм человека. Изучены эффекты освещения на конфликтное поведение и стрессовое состояние человека при использовании адаптивных систем освещения. Проанализированы методы и подходы разработанных адаптивных систем освещения и связанные с данными методами ограничения. Дано обобщение результатов теоретических и экспериментальных исследований. Предложен набор методов и средств для создания интеллектуальной многопользовательской адаптивной системы освещения.
Laushkina А.А., Roslyakova S.V., Smirnov А.V. Implementation of adaptive lighting systems to reduce stressful situations in multi-user spaces // Научный результат. Информационные технологии. - Т.5, №4,2020
Ключевые слова: освещение; адаптивные системы освещения; многопользовательские
системы освещения; интерактивное освещение.
INTRODUCTION
The impact of light on a person is varied, complex, and all-encompassing. Light affects vision, determines the cycles of daytime and seasonal functioning of the body, affects the levels of psychophysiological activity and mood [1-5]. The so-called non-visual effects of light were discovered [6]. The non-visual effects of light appear by stimulation of the eye photoreceptors. The receptors, rods and cones, are relatively well researched and described in the existing publications. The research over the past 25 years has shown that there is another type of photoreceptor in the eye. That type is important in the non-visual effects of light on humans. Discovered in 2002, these photoreceptors are known as photosensitive retinal ganglion cells (ipRGC), and their photosensitivity is based on the photopigment melanopsin [7]. The non-visual effects of light affect our mood, concentration, vitality, focus, mental and physical performance. There are three types of light effects on human body: circadian, cognitive, and emotional [8]. The result of these effects depends on time and duration of exposure to light, as well as on lighting quality [9].
Education at school and university is accompanied by anxiety and serious stress. Escalating behavior is reported by the media almost daily. Escalating behavior is an intensely developing conflict in time and emotional background. Individuals or groups of people show aggression, which often leads to verbal or physical attacks. The escalation of conflict usually includes narrowing and decrease of attention and progressive cognitive disfunction. Scientists have studied the effect of lighting on student anxiety and their aggressive behavior using different illuminance and color temperature of lighting [10]. The findings showed that different lighting modes could directly reduce student anxiety and improve their social behavior.
It is logical to assume that the effects already identified in everyday and work environment and the developed lighting systems are applicable to the educational environment [11].
Lighting has become a significant part of the "smart home", allowing to support various activities that require changes in lighting parameters (color temperature, illuminance level and luminance) in the process of learning, work, recreation, and communication between a group of people [10, 12, 13, 20]. The demand for smart lighting systems is confirmed by market growth; according to market research by the MarketsandMarkets, the smart lighting market will grow from $ 13.4 billion in 2020 to $ 30.6 billion by 2025 with a CAGR of 18.0% over the predicted period [14].
Adaptive lighting systems (ALS) are equipped with sensors and actuators that can dynamically adapt to current environmental conditions and people's needs. The consumer's ability to interact with lighting refers to the concept of "interactive lighting". Such systems foresee the aims and intentions of the user, which can be determined from the actions (video surveillance system and audio recording) or information about the environment (for example, light, pressure and noise sensors). On the one hand, adaptive lighting can react to user actions without direct interaction. On the other hand, adaptation can also be performed by the user intentionally. It is necessary to receive feedback from the system to create stable lighting parameters: the user performs various actions; the adaptive system reacts to this and changes the lighting parameters by the specified conditions. However, at the moment, the main problem is the lack of a mechanism for adapting the corresponding lighting systems to conflict behavior and stressful states of users in multi-user spaces.
The paper aimes to describe the methods and mechanisms to adapt the corresponding lighting systems to influence conflict behavior and stress state of users and related features.
MA TERIALS AND METHODS
The analysis of the articles was performed based on the existing methods and the means of adaptive lighting systems functioning. The following criteria were identified to choose methodology for ALS creation:
1. Set of sensors required for the ALS energy-efficient functioning.
Laushkina А.А., Roslyakova S.V., Smirnov A.V. Implementation of adaptive lighting systems to reduce stressful situations in multi-user spaces // Научный результат. Информационные технологии. - Т.5, №4, 2020
2. System autonomy (minimal user intervention to maintain the quality of the system).
3. Multimodal information about the external and internal environment (for a complete and accurate assessment of the environment with a further ALS adaptation of the).
4. User group focus (considering conflicts of preferences in the choice of lighting).
5. Consideration of conflict behavior and stress state of users.
6. ALS Intelligence (self-learning based on neural networks).
7. Combination of ALS monitoring and control (processing and output of multimodal information in adaptive lighting systems to create control protocols and self-optimization of multiuser systems).
At the moment, adaptive systems that respond to human activities, climate change, and daylight hours are quite common. However, the lighting settings that affect the person (color temperature and luminous flux of the luminaires) are changed by a given scenario or adjusted by the user [10, 12]. Adaptability in such systems is usually associated with dimming to save energy. For example, a neural network controller, which can regulate the lighting in a class considering illuminance of the environment and the number of people, was developed and tested [3].
However, in the smart home market, especially in the smart lighting market, there are no examples of systems that collect, process and transmit information about a group of users. At the same time, a large number of patents in the field of human biomonitoring testifies the relevance and prospects of combining monitoring and control of the lighting system in one device.
The patents describe lighting systems that work with wearable sensors that detect body temperature, momentum and user location in combination with external environmental factors such as temperature and lighting [8].
The existing solutions are aimed at reducing conflict situations in the choice of lighting modes in multi-user interaction. The patent [15] proposes a conflict resolution option based on a dynamic change in the user's position in space and the direction of his gaze, while the light characteristics output by the system show averaged values. It should be noted that this system does not allow making changes by the user, and also does not show information about changes made by other users. By depriving the user of this opportunity and by allowing the systems control lighting, there is a risk of a "loss of control" effect for the user, which will cause internal (and then external) problems.
Data storage, processing, and visualization are getting better lately. In addition to saving energy, lighting manufacturers are striving to improve perception and simplify lighting control. Therefore, more researchers are paying attention to improve-control algorithms [16-18]. Statistics, data modeling, and machine learning techniques are used to save energy. Among other things, an intelligent lighting system was developed with the possibility of continuous learning [19]. It can dynamically adjust illuminance, color temperature and luminance distribution according to specific learning requirements and conditions, such as the number of students per class, their academic performance, class schedule, room furnishing and energy efficiency. This system uses not only manual control but also automatic scene switching via integration with school timetables.
The authors suggested that the system they developed could help researchers study the correlation between student performance and environmental parameters.
The input data used in the existing studies included various combinations of indoor user placement and sensor information such as space occupancy and ambient lighting, schedule information, and user preferences. The authors faced the problem of integrating several systems and datasets into ALS to combine various information to select a suitable lighting mode. The researchers from the University of Waterloo have found a new approach to solve this problem: they used the advantages of big data and machine learning to increase the speed of data processing and the possibility of developing a system based on the information received.
J—J J"~J J^) J J/J Laushkina А.А., Roslyakova S.V., Smirnov A.V. Implementation of adaptive lighting systems
to reduce stressful situations in multi-user spaces // Научный результат. Информационные технологии. - Т.5, №4, 2020
RESEARCH RESlTlTl
r Start
▼
Get Course
Information
—course \
—.Changed?^-^" No
>
Switch to
Corresponding
Lighting Mode
1
Report Status
Fig. 1. Block diagram of switching lighting modes with class schedule
Figure 2 illustrates the mechanism and workflow of the proposed framework, which consists of two main processes: control flow and self-optimization flow.
Fig. 2. Structure of a self-optimizing classroom lighting environment
Such intelligent lighting systems have the potential to become a platform for future research on the effects of lighting on humans and can also be widely used in everyday life, which gives this approach an advantage over the existing ALS. However, from a technological point of view, the system still has many opportunities for improvement: the addition of devices to monitory environmental conditions, users' needs, and their conflict behavior and stress state.
RESULTS
It is possible to automate ALS, integrate intelligence properties and obtain reliable results at the end of the data processing cycle, combining the entire transformed set of signals and artifacts of the human body, that reflects the internal changes in the psyche and consciousness of people. The best way to collect multimodal information about the environment, changes in the conflict behavior and stress state of a group of users is via audio and video channels. Students work at computers with cameras already built-in, which will further facilitate the integration of ALS into office spaces.
The data is recorded continuously and dynamically, which helps to avoid false or approximate interpretations and conclusions arising from the statics of the image. Instant image fixation is provided
Laushkina А.А., Roslyakova S.V., Smirnov A.V. Implementation of adaptive lighting systems to reduce stressful situations in multi-user spaces // Научный результат. Информационные технологии. - Т.5, №4, 2020
with the ability to measure individual trainees' facial expression traits, reflecting various emotional and mental states of the person. The method of assessing the condition using audio and video channels is safe and painless for users. Based on the data obtained from the audio and video channels it is proposed to create a matrix of conflict behavior and stress states of people. This matrix is planned to be imposed on the matrix of the developed light modes that correspond to basic psychoemotional states, thereby teaching the system. It is suggested to use the following lighting modes:
1. "Standard" 300 lux, 4000 K
2. "Focus on the board" 1000 lx, 4000 K and darkened audience lighting 300 lx, 3800 K
3. "Board" 300 lux, 4000 K classroom lighting is completely off
4. "Concentrate" 1060 lux, 5800 K
5. "Activation" 675 lux, 6500 K
6. "Relaxation" 325 lx, 3500 K
7. "Maximum relaxation" 275 lux, 3500 K.
Light modes in Intelligent Multiuser Adaptive Workspace System (IMAWS) are programmed based on luminaires with two types of LEDs with standard values of correlated color temperature (2700 and 6000 K). In the fu-ture, this will simplify the integration of the developed ALS with luminaires of any manufacturers. ALS offers the optimal lighting mode considering the dynamically changing environment and user states. The described ALS will work according to the scheme in Figure 3.
Fig. 3. Functional diagram of the developed IMAWS
1 - microprocessor control unit;
2 - speaker;
3 - microphone;
4 - projector;
5 - dome video camera;
6 - directional video camera;
7 - Wi-fi module;
8 - microcontroller;
9 - room light sensor;
10 - room temperature sensor;
11 - indoor humidity sensor;
12 - indoor carbon dioxide sensor;
13 - LD display;
Laushkina А.А., Roslyakova S.V., Smirnov A.V. Implementation of adaptive lighting systems to reduce stressful situations in multi-user spaces // Научный результат. Информационные технологии. - Т.5, №4, 2020
14 - partition control module;
15 - shutter control module;
16 - light control module;
17 - lamp driver;
18 - LED lamps;
When the trainee is in the classroom at the workplace, multi-modal information necessary for further psycho-emotional assessment of the trainees is collected by the microprocessor control unit 1, using the dome video camera 5, directional video camera 6 and microphone 3. Abiotic environmental parameters in the room are also accumulated with the data obtained from sensors of illumination 9, temperature 10, humidity 11, carbon dioxide 12. The data obtained are analyzed in the microcontroller 8 and transmitted through the Wi-Fi module 7 by wireless connection to the microprocessor control unit 1, and duplicated by wired connection in order to improve noise immunity and stability of the system from external influences.
After receiving data from the microprocessor 8, the microprocessor control unit 1 makes the system analysis with the help of the algorithms in the unit memory, and gives the command to the executive modules, namely to the partition control modules 14, shutter control modules, and light flux control modules 16. The command received from the microprocessor control unit 1 is transmitted to the actuating elements through the microcontroller 8 with the help of the Wi-Fi unit 7 and is duplicated by the wired connection.
Upon receipt of the executive command from the microprocessor control unit 1 to the partition control module 14, the separation of the workspace by raising or lowering the acoustic partitions, thereby personalizing the workplace depending on the preference of the trainee.
When receiving the executive command from the microprocessor control unit 1, the blinds control module 15 regulates the natural light entering the classroom. The light control module 16 works together with it, the control signal from it is formed on the basis of the executive command received from the microprocessor control unit 1, then it is transferred to the lamp drivers 17 and goes to the LED-lamps 18 by means of wire connection. Depending on the current psycho-emotional state of the trainee it is possible to change such parameters of artificial light as color temperature and luminous flux, thereby adapting to each trainee in the room.
DISCUSSION
Having analyzed articles and patents, recommendations were formulated for improving multi-user adaptive lighting systems:
1. Considering the advantage of big data and machine learning, it is possible to optimize and train multi-user adaptive lighting systems, which can then automate the entire process of interaction between the user and the lighting environment.
2. It is necessary to take into account the psychoemotional state of users based on multimodal information and the regulation of light characteristics that affect the level of conflict behavior and stress states of users.
3. Development and implementation of multi-user adaptive lighting systems should contribute to changing the environment not to disrupt the everyday life and work of other people. ALS will reduce the number of conflicts when choosing and setting various lighting modes. The system will help to:
1) compromise in the choice of lighting to resolve the current situation, as well as reduce the number of future conflicts,
2) compare activities and group activities by zones using light,
3) choose lighting mode as a result of negotiations.
It is important to note that the acknowledgment of conflicts and their resolution should be left to the discretion of the user.
An open issue when building intelligent adaptive lighting systems is control, taking into account user conflicts in lighting preferences. The problem of reading multimodal information about users, data
J—J J"~J J^) J J/J Laushkina A.A., Roslyakova S.V., Smirnov A.V. Implementation of adaptive lighting systems
to reduce stressful situations in multi-user spaces // Научный результат. Информационные технологии. - Т.5, №4, 2020
RESEARCH RESlTlTl
about the external environment, and tracking the dynamics of conflict behavior in a multi-user space and changes in stress states of individual users remains unresolved.
Having analyzed the developed adaptive lighting systems, features and limitations were identified.
These limitations can be eliminated using artificial intelligence methods - machine learning, computer vision and expert systems. To determine conflict behavior and stressful situations, it is proposed to analyze data from audio and visual communication channels based on:
1) audio data - by assessing the intensity, speed of speech, frequency of the main tone, voice harmonization, and identifying non-verbal sounds (clatter, sighs, pauses). It is planned to use a set of methods, including digital filtering, fast Fourier transform, pitch extraction, wavelet analysis of noise signals, and neural network classification.
2) video data - it is planned to use an algorithm for image detection based on one of the architectures of convolutional neural networks (YOLO, SSD, Faster-RCNN). To assess the change in the position of human limbs in space and time, it is proposed to use the OpenPose system, which allows to determine the coordinates of body key points in real-time. To localize people in the room-coordinate system, use the augmented reality markers of the Aruco family to calculate the room rotation matrix in relation to the image plane.
CONCLUSIONS
An analysis of scientific publications and data on the topic of non-visual (biological) effects of visible light on the human body in adaptive lighting systems was performed. Based on the analysis of methods and means of the developed adaptive lighting systems, their features and limitations were identified. A set of methods was proposed to analyze the psycho-emotional state of a group of users. To determine the user conflict behavior and stressful situations, it is proposed to analyze the data of audio and visual communication channels.
At the moment, this is the most optimal set of methods to analyse conflict behavior and stress states of a group with the subsequent integration of this information into ALS. Such systems will help to adjust light settings in real-time based on multimodal information about the environment and psycho-emotional state of users.
References
1. D. Pavlov, D. Ivanov, and V. Petrov, "Energy-efficient biodynamic lighting for use in scientific and educational institutions," Second Balkan Light Junior Conference 2019. (Balkan Light Junior), Plovdiv, Bulgaria, 2019, pp. 1-4, DOI: 10.1109 / BLJ.2019.8883566.
2. Byun, J.; Hong, I.; Lee, B.; Park, S. Intelligent household LED lighting system considering energy efficiency and user satisfaction. IEEE Trans. Consum. Electron. 2013, 59, 70-76.
3. Chen, Y.; Sun, Q. Artificial intelligent control for indoor lighting basing on person number in classroom. In Proceedings of the 2013 9th Asian Control Conference (ASCC), Istanbul, Turkey, 23-26 June 2013; IEEE: Piscataway, NJ, USA, 2013; pp. 1-4.
4. Barrett, P.; Davies, F.; Zhang, Y.; Barrett, L. The impact of classroom design on pupils' learning: Final results of a holistic, multi-level analysis. Build. Environ. 2015, 89, 118-133.
5. Smolders, K.C.H.J.; de Kort, Y.A.W. Bright light and mental fatigue: Effects on alertness, vitality, performance and physiological arousal. J. Environ. Psychol. 2014, 39, 77-91.
6. Статья D. M. Berson, F. A. Dunn, and M. Takao, "Phototransduction by retinal ganglion cells that set the circadian clock," Science, vol. 295, no. 5557, pp. 1070-1073, February 2002.
7. International Commission on Illumination Commission URL: http://cie.co.at/publications/position-statement-non-visual-effects-light-recommending-proper-light-proper-time-2nd (accessed 12.11.20).
8. S V Roslyakova et al 2020 Possibilities to integrate wearable biomonitoring sensors into adaptive lighting systems. IOP Conf. Ser.: Mater. Sci. Eng. 944 012029
9. Fabio, B.; Chiara, B.; Ornella, L.R.; Laura, B.; Simonetta, F. Non Visual Effects of Light: An Overview and an Italian Experience. Energy Procedia 2015, 78, 723-728.
10.N. Wessolowski, H. Koenig, M. Schulte-Markwort, C. Barkmann. The effect of variable light on the fidgetiness and social behavior of pupils in school. Journal of Environmental Psychology Volume 39, September 2014, Pages 101-108.
J—J J"~J J^) J J/J Laushkina А.А., Roslyakova S.V., Smirnov A.V. Implementation of adaptive lighting systems
to reduce stressful situations in multi-user spaces // Научный результат. Информационные технологии. - Т.5, №4, 2020
RESEARCH RESlTlTl
11. Barkmann, C., Wessolowski, N., & Schulte-Markwort //Applicability and efficacy of variable light in schools. Physiology & Behavior, M. 2012. №105(3), 621-627. doi:10.1016/j.physbeh.2011.09.020
12. Choi, K.; Suk, H.-J. Dynamic lighting system for the learning environment: Performance of elementary students. Opt. Express 2016, 24, A907.
13. M.E. Kompier, K.C.H.J. Smolders, W.D. van Marken Lichtenbelt, Y.A.W. de Kort, Effects of light transitions on measures of alertness, arousal and comfort. Physiology & Behavior (2020) [accessed 27.06.20].
14. Smart Lighting Market [Electronic resource] // Smart Lighting Market by Offering (Hardware: Lights & Luminaires, Lighting Controls; Software, and Services), Communication Technology (Wired and Wireless), Installation Type, End-use Application, and Geography - Global Forecast to 2025 [Feb 2020] URL: https://www.marketsandmarkets.com/Market-Reports/smart-lighting-market-
985.html?gclid=CjwKCAjwztL2BRATEiwAvnALcqOgHtzS5AmqLO3NDuyXJP5ZVjIH_4yMCZDuFa2Rd6IWw BV2Gj 6QkBoC-KcQAvD_BwE
15. Patent Netherlands № 2017111815, 05.08.2015. Resolution of conflicts// Patent of the Netherlands № 10.10.2018 Bulletin № 28. / Nolan Julian Charles, Laurenson Matthew John [etc.].
16. US Patent № 2011145306/07, 29.03.2010 Smart Lighting Control System // US Patent № 20.05.2013 Bulletin № 14. // Klasmann Donald Louis, Murphy Michael Sean.
17. The patent of the Russian Federation № 2018145895, 21.12.2018 System of adaptive functioning of light emitting devices // The patent of Russia № 22.06.2020 Bul. № 18. // R.K. Gereikhanov // Patent of Russia № 22.06.2020 Bulletin № 18.
18.Netherlands Patent № 2017110407, 31.08.2015 The way of management of the lighting system, computer software product, portable computing device and set of lighting system // Netherlands Patent № 03.10.2018 Bulletin № 28 / Mace-on Jonathan David, Shraibi Sanae [etc.].
19. Baoshi Sun, Qiaoli Zhang, Shi Cao. Development and Implementation of a Self-Optimizable Smart Lighting System Based on Learning Context in Classroom. International Journal of Environmental Research and Public Health (Received: 21 January 2020; Accepted: 11 February 2020; Published: 13 February 2020)]
20. Choi, K., Shin, C., Kim, T. et al. Awakening effects of blue-enriched morning light exposure on university students' physiological and subjective responses. Sci Rep 9, 345 (2019).
Laushkina Аnastasia Alexandrovna, student of Saint Petersburg National Research University of Information Technologies, Mechanics and Optics of Saint Petersburg National Research University of Information Technologies, Mechanics and Optics Roslyakova Svetlana Vitalievna, assistant, expert of Saint Petersburg National Research University of Information Technologies, Mechanics and Optics
Smirnov Andrey Vyacheslavovich, engineer of Saint Petersburg National Research University of Information Technologies, Mechanics and Optics
Лаушкина Анастасия Александровна, студент Санкт-Петербургского национального исследовательского университета информационных технологий, механики и оптики
Рослякова Светлана Витальевна, ассистент, эксперт, Санкт-Петербургский национальный исследовательский университет информационных технологий, механики и оптики
Смирнов Андрей Вячеславович, инженер Санкт-Петербургского национального исследовательского университета информационных технологий, механики и оптики
