CC BY
16https://doi.org/10.48417/technolang.2023.03.04 Research article
Artistic Creation in Virtual Space
Lingyan Zhang, Yun Wang (El), and Jiarui Liu China Academy of Art, 352 Xiangshan Village, Xihu District, Hangzhou, Zhejiang, 310000, China
Abstract
The purpose of this paper is to discuss the changes brought by artificial intelligence to future art creation and to explore how art creation in virtual space will unfold. The language, concepts, and ideas of art creation in the contemporary age are rapidly transforming. The further development of digital technology makes it possible to create in a virtual space, and we no longer need to rely on the presence of analogical tools to create excellent works. Virtual reality and augmented reality technologies have been able to bring the integration of human and virtual environments to a new level, and the language of art is being redefined. By sorting out the relationship between art creation language and symbols and the future transformation of art creation language brought about by new media, this paper focuses on the changes in four aspects of future art creation: the transformation of art creation's space, language, tools, and mode. Finally, this paper uses the practice of children's art education projects as an example to illustrate the form of art language expression in virtual space, and to provide a reference for the future art creation methods in virtual space.
Keywords: Art creation; Virtual reality; Augmented reality; Virtual space; Multimodal interaction
Acknowledgment This work was supported by the Design-AI lab of China Academy of Art (CAADAI2022A001).
Citation: Zhang L., Wang Y. & Liu J.R. (2023). Art creation in virtual space. Technology and Language, 4(3), 40-48. https://doi.org/10.48417/technolang.2023.03.04
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License
УДК 7: 004.8
https://doi.org/10.48417/technolang.2023.03.04 Научная статья
Художественное творчество в виртуальном пространстве
Лингиан Чжан, Юнь Ван (И) и Цзяруй Лю Китайская академия искусств, Сяншань, 352, район Сиху, Ханчжоу, Чжэцзян, 310000, Китай
Аннотация
Цель этой статьи - обсудить изменения, которые искусственный интеллект привнесет в будущее художественное творчество, и исследовать, как будет развиваться художественное творчество в виртуальном пространстве. Язык, концепции и идеи художественного творчества в современную эпоху стремительно трансформируются. Дальнейшее развитие цифровых технологий дает возможность творить в виртуальном пространстве, и нам больше не нужно полагаться на наличие аналоговых инструментов для создания превосходных произведений. Технологии виртуальной и дополненной реальности смогли вывести интеграцию человеческой и виртуальной среды на новый уровень, а язык искусства пересматривается. Разбирая взаимосвязь между языком художественного творчества и символами, а также будущую трансформацию художественного языка, вызванную новыми медиа, эта статья фокусируется на изменениях в четырех аспектах будущего художественного творчества: трансформация пространства, языка, инструментов, и способа художественного творчества. Наконец, на примере практики детских художественных образовательных проектов иллюстрируется форма выражения языка художественного творчества в виртуальном пространстве и дается ориентир для будущих методов художественного творчества в виртуальном пространстве.
Ключевые слова: Художественное творчество; Виртуальная реальность; Дополненная реальность; Виртуальное пространство; Мультимодальное взаимодействие
Благодарность: Работа выполнена при поддержке лаборатории Design-AI Китайской академии искусств (CAADAI2022A001).
Для цитирования: Zhang L., Wang Y., Liu J.R. Art creation in virtual space // Technology and Language. 2023. № 4(3). P. 40-48. https://doi.org/10.48417/technolang.2023.03.04
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License
INTRODUCTION
At this moment, content-generating AI has begun to sweep the world and the very process of art creation is undergoing unprecedented changes. Each creative era bears the mark of its history and of the technology that made it possible. The language, concepts and ideas of art creation in the age of artificial intelligence are transforming, and the further development of digital technology is making creation "in the void" possible. This expression means that we no longer need to rely on the presence of analogical tools and that we are able to generate a series of realistic digital works through one or multiple strings of code. The question is inevitably, how and how much digital technologies are affecting our creative process?
With the introduction of digital technology, the sector of digital art began to sprout, and the scene of art creation started to blur between the real and the virtual, and art creation in the real environment begins to shift to the virtual space, forming a two-dimensional mixed field of reality and virtuality (Xu & Hsu, 2020). In this situation, artists experimented with various media and forms of art creation and began to experiment with interdisciplinary joint creation. Various forms of creation started to emerge, such as computational art, experimental art, video art, etc. The very materiality of works also began to shift from physical products to digital collections.
Virtual reality technology is mainly formed by computer technology, multimedia technology and digital image processing technology. Wearable devices are used to convey images and to to simulate experience effects in virtual scenarios, allowing the users to interact in those environments in a multi-sensory way, provoking an immersive experience (Burdea & Coiffet, 2003). Unlike virtual reality technology, AR (augmented reality) does not fully immerse the user into the virtual environment, but uses computer vision, perception technology and graphics processing to superimpose virtual objects onto a real scene, enabling the user to perceive the existence of both reality and the virtual. It enables users to interact with virtual and real elements in real time (Carmigniani & Furht, 2011). With the help of virtual reality technology and augmented reality technology, art creation can unfold in an artistic atmosphere that is innervated with futuristic technology and interactivity, creating a gamified and immersive experience and providing users with the possibility of multi-sensory interaction in their understanding of art.
With the emergence of Transformer architecture, the field of image composition has left the era of the Generative Adversarial Network (GAN) and ushered in the combination of Natural Language Processing (NLP) and computer vision technologies to produce images that are more attuned to users' needs. Led by the trend set by the Artificial Intelligence Global Company (AIGC), many companies have gone on to create proprietary domain-based Large Language Models (LLM) based on deep learning architectures to handle a variety of natural language tasks. Using the LLM, various AI art creation tools have started to emerge. If in the past art creation was done with hands and physical tools, now the full-scale development of AI tools in daily life has led to a disruptive change in the instruments, methods and technical logic of the entire process. The very language of art is being redefined in the context of artificial intelligence (Mazzone & Elgammal, 2019). Thus, there is an urgency for us to think about how AI, as
a virtual tool, can organically link with concrete design activities and change the creation of art in physical space. This also leads us to think about how the way of making art will exist in the future.
This study will sort out the relationship between the language of art creation and his symbols, addressing the future transformation brought about by new media. To this end, we will illustrate the forms in which artistic expression manifest itself in the digital scenario, providing specific application examples. The aim is to discuss the changes brought by AI to art creation and to explore how it will unfold in virtual space.
ARTISTIC LANGUAGE AND SYMBOLS
Language, as a unique human tool used to express emotions and communicate ideas. Language is a symbolic system, a more complex system than any other, and every symbol can be used as a tool in a speech act. This means that language is not only a framework for action but is the medium of a specific form of activity, a "symbolic action" (Burke, 1966). During the long evolution of humankind, a common set of symbols, expressions, and rules of processing was developed and used to communicate, i.e., to exchange ideas, opinions, and thoughts. Language is the set of tools that allows people to express themselves in a variety of ways.
From the semiotic point of view, symbols are also the building elements of that specific form of communication that is the language of art. By its very nature, art language is composed in a form that serves to express emotions; it creates symbols that express human emotions. In calligraphy, for example, lines and strokes are the linguistic elements or symbols that serve this goal. In painting, style, color, and composition are the symbolic language that can express emotion. In art, the symbols are the concrete sensual basis of the bareness of thought; the symbols are the most essential elements of the artistic work, and the symbols constitute the expression of art. With the continuous evolution of art creation, the language of art creation also begins to become more and more abstract, conceptualized and standardized, and each art creator forms a unique toolset of languages and symbols.
In his 1994 book The Language of Displayed Art, Michael O'Toole (1996) explored the visual grammar of art, such as painting, sculpture, and architecture, and proposed three kinds of meaning: reproductive, modal and combinatorial meaning. Two years later, Kress and van Leeuwen published Reading Images: The Grammar of Visual Design, in which they discussed the interaction between visual modalities and the transmission of meaning through visual images in posters and advertisements. Kress and van Leeuwen proposed a theoretical study and analytical framework of visual grammar (Kress & van Leeuwen, 1996). They emphasize that this notion of "grammar" is different from the traditional sense. What they call "grammar" focuses on how elements combine to produce a meaningful whole. Building on the work of Kress and van Leeuwen, Clare Painter focuses on the visual narrative process of sequential images and provides a detailed categorization of visual grammar, moving from still images to a dynamic perspective that considers visual transformation in the context of the section that will follow the one observed, exploring the meaning of graphic interaction (Painter et al., 2013). The
interpretation of visual grammar depends on different artistic and cultural backgrounds and symbolic perceptions of images. Visual narrative grammar is based on the theory of multimodal discourse analysis, a breakthrough based on the systemic functional linguistics created by British scholar M.A.K Haliday, which breaks the boundaries of the field of linguistics (Haliday, 1973).
Multimodality is a concept that can help us understand the use of a symbolic theory of language as it is applied to art, both in physical space and in virtual space (Liao, 2019). Symbols transmit information by visual, sonic, and tactile means. Art has always been concerned with bringing together different modes of experience, combining various expressive codes of symbols (drawing, writing, sound, the very materiality of the components). In embracing virtuality, these modes widen and their interplay becomes increasingly complex, encompassing, e.g., even forms of direct and responsive interaction with the viewer, impossible in classical figurative art. Just by integrating the old representational forms within a digital context, we already have a double overlap of symbols (the traditional and the virtual). Thus the creative process of digital art arises and develops in a constitutive multimodality.
THE TRANSFORMATION OF ARTISTIC EXPRESSION
The medium, as the basis of language transmission, is the basic point of distinction between the various arts. The multimodal language makes the medium of art creation somewhat fluid among the various forms of expression, as it makes it much simpler to use multiple forms simultaneously. It is therefore undeniable that the introduction of new media is an important phenomenon in the development of contemporary art, denoted by the renewal of artistic methods and means, in a way that enables creators to gain a wider space for the expression of artistic concepts. The arrival of smart technology is also bound to bring a series of changes to art creation. We consider four aspects in which the transformation of the process of artistic creation becomes most prominent: space, language, tools, and mode.
1) Transformation of space. Virtual reality and augmented reality technologies have been able to bring the integration of human and virtual environment to a new level. With the development of this technology, some European and American painters have also started to use VR technology for creation, expanding painting from two-dimensional space to three-dimensional space. The change of medium brought by technology has shifted the space of art creation from real to virtual. The rise of concepts such as virtual space and meta-universe has made artworks less singular in dimension, which means a capability to work on previously inaccessible levels of space. The emergence of new media overtake the traditional space that the piece could occupy, and the spatial scope of artworks becomes more extensive and diversified. This breakthrough also makes the expression of artworks more open and interactive. By painting in this virtual environment, the work is no longer two-dimensional, but realizes a three-dimensional effect, and the artist can paint a work that can be displayed in 360 degrees, which can be said to have greatly enriched the artist's expressive language. For example, the Japanese new media
art team TeamLab has held large-scale interactive installation art exhibitions of virtual reality around the world.
2) Transformation of language. New forms of art language are beginning to emerge in virtual space. In the past, works created by human painters over years and months can now be generated by machines in just one second. In the works of Leng Jun, a hyper-realistic painter and a leading figure among the modern Chinese scene, the most prominent feature is the extreme similarity to photographs, especially in what concerns the human figures: hairs, pores, and even the clothing patterns. Since 1839, when the French painter Louis Daguerre invented the first camera, the traditional realism school of painting has been under unprecedented impact. Most people believed that the appearance of the camera made the strive for realism no longer necessary or justified (after all, the camera could capture reality much more faithfully than a painting). The emergence of the camera compelled artists to rethink the strenuous pursuit of realistic reproduction. Painters began to think about how to establish an independent language in their works, understanding that only through an original language they could leave a mark in art history. The focus thus shifted from fidelity to reality toward the enhancement of one's own expressive capability, a phenomenon confirmed by the immediate emergence of different painting schools such as Impressionism and Post-Impressionism. And that is just one possible example of how the development of technology affects the birth of new forms of art language. In light of this, if we consider the art language in virtual space, we can discover two main characteristics, not yet so prominent in the past: one is virtualness and the other is dynamism. With the help of special computer software and software and hardware facilities such as VR glasses and VR handles, art creators can create, on one hand, virtual elements in virtual scenes and combine them with real scenes to create an enhanced interactive experience. On the other hand, the art language that used to be presented statically on traditional paper and figures can be easily animated gaining a dynamic element that is not only that of performance, but that of a personalized interaction with the individual user.
3) Transformation of tools. Every form of innovation opens up a field of opportunity that allows for improvements in previous techniques and the addition of diversity to the creative process. The overflow of novel technical means always brings new possibilities to creation, whether it is virtual painting software or physical painting tools. But the revolution in the tools of art creation in virtual space is not only reflected in a widening of the forms of expression. The use of software and digital drawing technology is flexible and versatile, and the use of traditional techniques based on its framework is not limited or excluded. The new tools also present a much more open structure, as they permit a process of trial-and-error in a much less strict way. The emergence of intelligent tools of artistic expression has also completely transformed the matter of accessibility to artistic creation. The implementation of AI technology has meant drastic decrease at the entry threshold. Anyone can enjoy the fun brought by artistic creation, without investing the time necessary to acquire a minimal proficiency with the old analogical tools.
4) Transformation of mode. Multimedia artworks, as the name indicates, implements a variety of media forms such as audio, video, and images. The combination
of these forms allow for a more diverse expression of the work. The mode is not limited to content, but also to reception. For example, the speed of information dissemination on the Internet allows digital artworks to spread rapidly among the general public. In a special exhibition at Cambridge University, contemporary calligraphy artist Wang Dongling used the language of calligraphy to create a hyperspace dialogue with Stephen Hawking and Xu Zhimo. And in 2021, Wang Dongling also experimented with iPad creation, using VR technology to write Zhuangzi's A Happy Excursion again. He also tried to create The Way of Confucius through AR technology, using a motion capture suit to record the trajectory of his body's movement while writing and digitizing it (a procedure that he later extended to create several other digital works). Whereas in the past, a calligrapher needed to use both hands to master the brush skillfully for direct creation on paper, now he or she only needs to control the handle of a device to complete the creation in virtual space.
ARTISTIC LANGUAGE EXPRESSION IN VIRTUAL SPACE
With the emergence of big data and new digital technologies, art creation in virtual spaces has undergone disruptive changes. Shown below is the example of a children's practice during a project in art education (Figure 1), that is useful to illustrate how art creation in virtual spaces will unfold.
a b c Figure 1. Children creating art in a virtual space
Art education in the digital age has both a broader space for development and at the same time faces greater challenges. Children's art education is of course an important part of fostering children's overall development. The project wanted to implement digital technology in the field of childhood learning with software and hardware product design integrated into this innovative model of art education. Figure 1-a shows the child in the process of selecting the form of the object that will then be shaped in the virtual space; Figure 1-b shows the process of sculpting the original model after the child puts on the AR glasses; Figure 1-c shows the process of coloring and haloing the sculpted model. This project is designed to be highly expandable in the future through the innovation of hardware products and tools centered on children's ability to learn many art disciplines,
enabling children to interact and learn about art creation in a more immersive way, and empowering art education to be better integrated into the life of the new generation. This solution opens up new opportunities for children's art education in virtual space with the multimodal product design of augmented reality tools.
CONCLUSION
The combination of technology and art is already opening up new possibilities for the future of art. In the era of intelligent generation, the iterative renewal of science and technology means the continuous transformation of contemporary art, and traditional art needs to make innovations in the context of such development and integrate and progress with technology. By discussing the relationship between art creation language and symbols, the current art creation space is shifting to a virtual space and a mixed space of virtual and real. The art creation language is revealing its virtual and dynamic characteristics, and the change of art creation language makes the space for expressing our art concepts freer and broader. The final example showed how the potential of digital art also possesses immense possibilities for pedagogy. The ease of access, the simplification of the modes of engagement, along with the possibility of employing any space, open up the opportunity to introduce the child to the language of artistic creation in an immediate and fun way.
REFERENCES
Burdea, G., & Coiffet, P. (2003). Virtual Reality Technology. Presence: Teleoperators and Virtual Environments, 12(6), 663-664.
https://doi.org/10.1162/105474603322955950 Burke, K. (1966). Language as Symbolic Action; Essays on Life, Literature, and Method.
University of California Press. https://doi.org/10.1525/9780520340664 Carmigniani, J. & Furht, B. (2011). Augmented Reality: An Overview. In B. Furht (Ed.), Handbook of Augmented Reality (pp. 3-46). Springer. https://doi.org/10.1007/978-1-4614-0064-6 1
Haliday, M. A. K. (1973). Explorations in the Functions of Language. Edward Arnold. Kress, G. & Leeuwen. T. V. (1996). Reading Images: The Grammar of Visual Design. Psychology Press.
Liao, Q. Y. (2019). Modal Symbols Functioning as the Base of Propositional Symbol. Journal of Foreign Languages, 42(5), 44-52.
http://j fl.shisu.edu.cn/EN/Y2019/V42/I5/44 Mazzone, M. & Elgammal, A. (2019). Art, Creativity, and the Potential of Artificial
Intelligence. Arts, 5(1), 26. https://doi.org/10.3390/arts8010026 O'Toole, M. (1996). The Language of Displayed Art. Leicester University Press. Painter, C., Martin, J. R., & Unsworth L. (2013). Reading Visual Narratives: Image Analysis of Children's Picture Books. Equinox Publishing.
Xu, R. & Hsu, Y. (2020). Discussion on the Aesthetic Experience of Artificial Intelligence Creation and Human Art Creation. In H. Shoji (Ed.), Proceedings of the 8th International Conference on Kansei Engineering and Emotion Research. KEER 2020. Advances in Intelligent Systems and Computing (vol 1256. pp 340-348). Springer. https://doi.org/10.1007/978-981-15-7801-4 36
СВЕДЕНИЯ ОБ АВТОРАХ / THE AUTHORS
Лингиан Чжан, [email protected] Юнь Ван, [email protected] Цзяруй Лю, [email protected]
Lingyan Zhang, [email protected] Yun Wang, [email protected] Jiarui Liu, [email protected]
Статья поступила 2 июня 2021 Received: 2 June 2021
одобрена после рецензирования 6 августа 2023 Revised: 6 August 2023
принята к публикации 4 сентября 2023 Accepted: 4 September 2023
