CC BY-NC-ND
41
DOI: 10.17323/2587-814X.2021.3.7.23
Integrated neurosymbolic decision support systems: problems and opportunities
Alexander V. Demidovskij ©
E-mail: ademidovskij@hse.ru
Eduard A. Babkin
E-mail: eababkin@hse.ru
National Research University Higher School of Economics
Address: 25/12, Bolshaya Pecherskaya Street, Nizhny Novgorod 603155, Russia
Abstract
The current problem of developing new kinds of decision support systems for different categories of management personnel is addressed in this study. A critical feature of such systems is their distributed and decentralized nature, which enables the construction of next-generation information systems in the form of Multi-Agent Systems, Internet of Things, or Fog Computing Architectures. Parallel models of the dynamics of artificial neural networks are produced under such realistic circumstances, demonstrating their potential for addressing a variety of issues. The purpose of this study is to conduct a critical analysis of the problem of integrating Artificial Neural Networks with decision support systems using a corpus of relevant scholarly literature. To tackle this question, the Design Science Research methodology was considered. According to this methodology, a literary search strategy was established, scientific literature was collected and analyzed, and key comparisons between different solutions were emphasized. The study resulted in the presentation of the most important findings, outstanding issues, and potential areas of fundamental and applied solutions. A consistent trend toward the development of decision support systems based on integrated neural-network methods has been observed, which is efficient and cost-effective since it enables the creation of distributed and trainable decision support systems.
Key words: linguistic decision-making; multi-criteria decision-making; decision support systems; neurosymbolic systems; artificial intelligence.
Citation: Demidovskij A.V., Babkin E.A. (2021) Integrated neurosymbolic decision support systems: problems and opportunities. Business Informatics, vol. 15, no 3, pp. 7—23. DOI: 10.17323/2587-814X.2021.3.7.23
Introduction
Fundamentally new conditions of information processing, planning and strategic management in digital enterprises based on the self-organization principles necessitate the implementation of such flexible organizational forms as virtual [1], agile [2] and distributed [3] enterprises. The pace of production processes and the frequency of process modifications are substantially increased in such companies. As a result, at the current stage of widespread digi-talization, the need for developing new kinds of decision support systems (DSS) for different categories of management personnel is increasing. At the same time, the distributed structure of such systems becomes an essential characteristic of new types of DSS. This can be attributed to several factors.
One of these critical factors is the ongoing development of a variety of massively parallel and distributed computing technologies and communication solutions within the context of the business information technology environment. This enables the practical viability and economic feasibility of developing new generations of information systems on the basis of multi-agent systems (MAS), Internet of Things networks, and so-called Fog computing architectures [4]. The aforementioned distributed computing infrastructures can include millions of autonomous components interacting in a decentralized manner under dynamic and software-defined networking.
Under these circumstances, certain components of computing infrastructures (for example, neurons as agents) may be directly linked with the structural elements and communication algorithms employed in parallel models of the dynamics of artificial neural networks (ANN) [5—12]. Indeed, various researchers [13—16] agree that a single device (in Internet of Things networks, in Fog computing infrastructure) or a distinct intelligent agent (in the MAS) performs the function of a separate neuron or implements a particular fragment of a global multi-level neural network. In this case, the ANNs' dynamics can be effectively implemented utilizing shared memory or message transmission methods.
Consequently, it is logical to assume that the neural network method is the primary path for distributed DSS development and implementation. The integration of the artificial neural networks into the DSS is critical and promising since it facilitates the practical use of symbolic and sub-symbolic computations simultaneously. The integration of the ANNs with various systems has existed over a long period of time and is referred to as neurosymbolism [17].
The purpose of this study is to conduct a critical analysis of the issue of artificial neural networks and decision support system integration based on current scholarly literature. Design Science Research Methodology (DSRM) was selected to resolve this problem, as it establishes the fundamental criteria for the strategy of systematic collecting and analysis of relevant scholarly literature, as well as the format of the results obtained [18].
In accordance with this methodology, the approach for conducting the literary search was established, the scholarly literature was collected and analyzed, and the primary criteria for comparing alternative solutions were defined. The analytical results made it possible to identify the most important findings, open issues, and prospective areas of fundamental and applied research.
This article consists of four sections. The first section (Foundations) examines the distinctions between symbolic and connection-ist approaches to information processing, as well as the DSS and neural networks features. The second section (Methodology) covers the fundamental principles of conducting research using the DSRM methodology and briefly describes the process and outcomes of systematic literature collection and analysis. The third section (Analysis results) contains conclusions on the role and the use of neural-network DSS in decision-making support processes and identifies the most promising topics for further research. The final section outlines the study findings.
1. Foundations 1.1. Decision support systems
The necessity of developing decision support systems (DSS) is determined by the increasing complexity of decision-making in a rapidly changing environment, the expanding amount of data, and the growing number of interconnected factors affecting the decision-maker's actions. The DSS is an interactive information system that utilizes the data and decision-making models to provide users with convenient and efficient access to information resources and a range of information processing and decision-making opportunities [19-20].
There are many different classifications of DSS. The most suitable categorization criteria within the framework of this study is the
type of the dominant technology component or driver of the decision support system [21]. The following types of DSS are distinguished according to this classification:
1. Data-driven DSS. The key element of such DSS is the analysis of large volumes of structured data. An example of such systems is a variety of data warehouses. Frequently, such DSS are used in combination with Online Analytical Processing (OLAP) systems [19, 22].
2. Model-driven DSS. Computational models, the parameters of which are determined by the DSS, constitute an essential part of such DSS. An example of such systems is the DSS that predicts the demand and pricing for goods.
3. Knowledge-driven DSS. These DSS assist the decision-maker by utilizing business rules and knowledge bases. An example of such systems is the expert systems.
4. Document-driven DSS. The main component of such systems is unstructured data, and their primary function is to assist the decision-maker in finding the required materials. An example of such systems is search engines.
5. Communications-driven and Group DSS. These DSS are designed to assist a group of decision-makers in the problem solving process. Such systems include integrated collaboration solutions: document sharing and editing systems, video-conferences, chats, etc.
It is important to note that a significant part of contemporary DSS is hybrid. Modern DSS include several key technical components simultaneously, such as data, models and knowledge. In this case, the lack of data during the DSS development stage may be compensated by business rules and predefined scenarios. Moreover, the steady accumulation of data will enable the use of OLAP technol-
ogy for data analysis and the creation of new mathematical models to provide more effective suggestions for decision-makers.
As will be described further, there is a trend towards incorporating artificial intelligence (AI) techniques, such as artificial neural networks (ANN) and case-based reasoning (CBR) into DSS, since these techniques make it possible to enhance the predictive capability of the DSS and discover hidden patterns within the given data [23]. Recently, DSS incorporating AI components have been called intelligent DSS (IDSS). Analysis of the scholarly literature reveals that a substantial amount of research in the field of decision support systems is dedicated to the problem of multi-criteria decision-making (MCDM). For instance, at the beginning of this research (January 2021), the Scopus scientific citation database included 3,516 papers with the keywords "multi-criteria AND decision AND decision support systems." When developing an IDSS for MCDM, it is important to take into account the uncertainty in information and knowledge about the issue, as well as to use different techniques to reduce uncertainty, for example, by introducing convolution or ranking contradicting criteria and expert assessments [24]. Furthermore, the decision-making process can be iterative.
Thus, in addition to the traditional requirements for the DSS, such as rapid development using standard software building blocks, fast deployment of the DSS and the reliability and accuracy of its recommendations, a number of additional criteria must be formulated. Firstly, it is necessary to ensure high performance of the DSS to provide the decision-maker with the necessary real-time assistance. Secondly, the DSS must be capable of learning. In an ideal scenario, the DSS should also be capable of real-time training on data appearing in the system. Thirdly, the DSS should be capable of operating with a priori ambiguity of context, knowledge and logic. This is especially
important for ill-structured problem solving and incorporating intelligent computer-aided design systems [25]. Finally, any DSS should be interpreted and comprehended by both the decision-maker and other stakeholders. This problem is particularly acute for DSS systems that use artificial neural networks, the work and decisions of which are usually difficult to interpret.
Taking into consideration the aforementioned criteria, the development of the DSS should be carried out according to the hierarchical approach [25], when each level has its own models and the decision-making model is made up of cyclically repeated stages:
♦ collecting all types of information (clear and fuzzy);
♦ data analysis;
♦ data conversion;
♦ development of criteria for evaluating decisions;
♦ producing decisions (alternatives);
♦ analysis of alternatives and selection of a subset of options (or one of them) based on the specified criteria.
Separately, we should highlight the requirement for the interpretability of findings generated by contemporary systems with AI elements. This requirement is formed at the state and international organization levels. For instance, Canada's Directive on Automated Decision Making aims to "ensure that automated decision systems are deployed in a possibly risk-free manner and lead to <...> interpretable decisions <...> providing the explanation of the system's decisions" [26]. According to the principles of the Organization for Economic Co-operation and Development, which includes 38 countries, "There should be transparency and responsible disclosure around AI systems to ensure that people understand AI-based outcomes and can
challenge them" [27]. Thus, modern DSS, which are heavily reliant on AI components during the development stage, fall under the regulation of these and many other legislative documents. This, in turn, highlights the issue of development approaches for the IDSS, the search for a balance between the use of AI elements to enhance the stability and predictive abilities of the DSS, and the maintenance of a high-level interpretability of the produced results.
1.2. Symbolic and connectionist approaches
One of the primary constituents of DSS development is knowledge representation and modeling, as well as the ability to analyze information and draw conclusions based on existing knowledge. The way knowledge is represented and modeled by the human brain currently determines two different currents of scientific thought.
The symbolic school of thought holds that the human cognitive function is founded on the representation of information through symbols, which are subsequently analyzed and, if necessary, split into smaller components [28]. Symbols can be split or combined into other symbols. From the perspective of symbolic computations, human cognition is a case of symbol manipulation. Simultaneously, formalizing and implementing computing operations on symbolic relations appears to be quite challenging.
On the other hand, sub-symbolic systems rely on an abstraction such as a neuron [18]. This approach is based on the notion that the human brain is a massive natural neural network that operates on a distributed representation of information and is capable of performing meaningful operations on it. Sub-symbolic systems are constructed from fundamental building blocks - neurons that operate in parallel and store only a partial representa-
tion of the input signal in their state at each moment of time. Then the general representation of the input impulse, stored and modified by ANN, is distributed. This approach to knowledge representation and operations is referred to as sub-symbolic, or connectionist.
It should be emphasized that these two paradigms coexist due to a set of reciprocal benefits and drawbacks. For example, sub-symbolic systems have the ability to learn and adapt to changing data. Symbolic systems lack such autonomy and therefore require manual coding of internal rules. The latter may result in the symbol grounding problem, when the system programmer's viewpoint is reflected in the rules that the system employs. Sub-symbolic approaches are distributed by their nature and have high efficiency and reliability. In addition, they contain a crucial learning component, allowing us to train sub-symbolic methods for solving a specific task and constantly modify them as new information appears. At the same time, this approach is difficult to interpret. For this purpose, a separate field of knowledge called explainable AI (xAI) is being developed. This field is devoted to the creation of techniques for extracting meaning from complicated computational models.
Another disadvantage of sub-symbolic methods is their inability to formulate complex relationships due to the limitations of modern methods of distributed knowledge representation [28-29]. Certain advancements in this area have enabled the solution of a variety of distributed representation issues associated with sub-symbolic systems. These include Tensor Representations [30] capable of representing recursion in symbolic structures, further development of ideas concerning more compact distributed representations via holographic reduced representations (HRR) [31], and other methods of symbolic calculations. As a rule, this class of methods is called vector symbolic architectures (VSA) [32-33].
Contemporary scholarly literature dedicated to the research and development of concepts for creating a strong or general artificial intelligence emphasizes the need to develop integrated solutions that would be based on both deep learning and symbolic reasoning [18]. This is due to the fact that a system including the elements of a general AI must be semantically justified, interpretable, and reliable, and its solutions must be completely trusted by all concerned parties. The above-mentioned characteristics of symbolic and connection-ist approaches, as well as the concept of a general AI, demonstrate that the creation of a general AI is unachievable without the mutual integration of these approaches and the development of neurosymbolic techniques. As previously stated, a legal framework is being established, where it is clearly indicated that the results must be fully interpretable, which becomes a prerequisite for the system's deployment. These actions complicate the implementation of solely connectionist approaches, since it is often impossible to explain how the neural network produced the suggested decision. Thus, even the implementation of systems with narrow AI components requires the use of symbolic methods in addition to neural network methods, making the task of designing and implementing neurosymbolic methods even more critical.
The degree to which the two paradigms are integrated varies significantly in the process of developing neurosymbolic systems. According to the Kautz taxonomy, such systems are classified into six categories [34] (the author's names of system types are retained):
Type 1: Traditional Deep Learning based systems that take symbols as input (for example, a text with a question) and output symbols as a result (a text with an answer to the question).
Type 2: Symbolic[Neuro]. These systems combine symbolic reasoning (for instance, selecting an action while driving an autonomous vehicle) with ANN to solve a specific task (for example, detecting objects on the road).
Type 3: Neuro-Union-compile[Symbolic].
These neural network systems utilize symbolic rules as the input (for example, the integral mathematical expression), and generate symbolic rules as the output (for example, the solution of the mathematical problem of calculating the integral).
Type 4: Neuro-Symbolic. This type represents a cascade of sub-symbolic and symbolic systems, where the symbolic system is also capable of learning (for example, Neuro-Sym-bolic Concept Learner [35]).
Type 5: Neuro[Symbolic]. Symbolic reasoning is performed at the sub-symbolic level (there are no examples of such systems yet).
Thus, while developing a DSS with AI components, the use of both symbolic and sub-symbolic approaches is inevitable. Depending on the tasks and requirements for the final DSS, different degrees of integration of sub-symbolic approaches may be necessary. Simultaneously, the DSS cannot become entirely symbolic due to the requirements for interpretability of the system's results and the necessity of establishing a priori decision-making rules.
We will further examine existing DSS and conduct an analysis of the subject area's development trends. Then, we will formulate suggestions for creation of neurosymbolic DSS based on the neurosymbolic paradigm's key components. In particular, we will examine how to simultaneously represent knowledge in distributed and symbolic forms, as well as how to convey symbolic reasoning at the sub-symbolic level.
2. Principles of research according to the DSRM methodology
The subject of this research is the development of new models and algorithms for distributed neural-network DSS. Such studies belong to the field of engineering-oriented sciences about artificial intelligence. In this subject area, it is common to use the Design Science Research Methodology (DSRM) [19, 36-40].
For conducting scientific research in the area of business informatics using the DSDM methodology, it is essential to develop a strategy for the systematic collection and analysis of relevant scholarly literature describing different IT solutions to the examined problem [36-40]. The purpose of systematic collection and analysis of relevant scholarly literature is to determine as accurately as possible the research problem and stimulating factors (relevance, unresolved issues, the emergence of new technologies).
The international scientific community has developed a set of guidelines and a pattern for defining the strategy of literature analysis. This study utilizes the most exhaustive and detailed methodology proposed in [40]. This methodology allows us to fully justify the criteria for selecting literature for analysis, objectively present the results of the analysis and highlight the key reasons for the need to conduct one's own research and propose new developments (in our case, to justify the need for fundamental and applied research in the field of distributed DSS based on the neuro-paradigm).
The strategy of systematic search and analysis of literature [36, 40] is based on the division of all scholarly articles into two categories: configurative and aggregate publications. Configurative articles assist in determining the circumstances in which a particular artifact displays its useful quality.
Aggregate articles help in determining which artifacts were most often utilized to solve the problem. The requirement for the creation of a corpus of articles, which should include both configurative and aggregate articles, establishes the necessary terms for validating the hypotheses regarding the applicability of newly developed artifacts in the conditions of the studied problem situation, and also provides arguments in favor of the functionality of the solution under development. The division of scientific articles into two categories corresponds to the definition of two types of new results that need to be identified and analyzed. These two types of findings are referred to as construction heuristics and expectation heuristics, respectively [8]. Construction heuristics are the requirements for proper functioning of the internal environment of an artifact in accordance with the external environment. Internal mechanisms and their structure are disclosed at this stage, taking into consideration the intended effect on the external environment. Additionally, the heuristic includes specific knowledge that may be used in the future to create new artifacts. Expectation heuristics are the definition of the limits of applicability of an artifact, the conditions for its usage and the cases in which it will be useful. The heuristic defines the artifact's external environment, i.e. the context in which it may be used and the limits of its performance. The knowledge created at this stage can be applied to design and develop new artifacts.
Researchers of the artificial intelligence field suggest structuring the results of search and analysis of literature in the form of two analytical tables. In the first (intermediate) analytical table the information is presented in the context of each individually analyzed work. For each work, the key issue, the article's category, the article's most important construction heuristics, the final outcome of applying the suggested heuristics, and met-
rics for comparing the article's findings to one's own solutions are specified.
The second (final) analytical table allows us to create an aggregated representation of the results from the literature analysis in relation to the re-examined issues. This gives the possibility to justify the choice of problems and the methods used in one's own research.
Taking the aforementioned methodological principles into consideration, the following strategy for searching and evaluating literature on the subject of distributed neural-network DSS was developed for this research:
♦ search period — 1991—2021 (the period when the most significant fundamental and applied works in the field appeared);
♦ sources of information — citation databases Scopus and Web of Science (publications on the research topic are most fully indexed in these citation databases in the sections Computer Science, Decision Sciences and Management Information Systems)
♦ search and works language — English
♦ search keywords — "decision support systems", "intelligent decision support systems". The first set of keywords is more general in relation to the second; its usage is justified by the limited number of publications that may be located in citation databases using the second (more specialized) set of keywords. In particular, the search of the second (more specific) set without applying exclusion criteria resulted in 300 works in the Web of Science database and 1269 works in the Scopus database. At the same time, using the first (more general) set returned 20,003 works in Web of Science and 108,131 works in Scopus.
♦ criteria for excluding articles from the corpus:
— low relevance, as assessed by the article's abstract review;
— the absence or difficulty of reproducing the article's results;
— the complexity of reusing software artifacts presented in the article;
— the inability to analyze the article due to the use of a language other than Russian and English;
— the absence of the use of artificial intelligence elements (such as fuzzy sets or ANNs) in the construction of the proposed methods.
As a result of implementing the proposed strategy, the following corpus of articles was formed (Table 1). The corpus was divided into five years intervals, starting from 1991.
Table 1.
The structure of the corpus of articles analyzed
Search Aggregate Configurative
period publications publications
1991-1995 6 7
1996-2000 6 8
2001-2005 7 7
2006-2010 8 28
2011-2015 7 20
2016-2021 19 34
The total number of articles selected for analysis is 157. We took into account that the initial number of works satisfying the selected search query in the Web of Science citation database was 303. However, by applying the exclusion criteria to the works in both citation databases (Scopus and Web of Science), the number of works for analysis was reduced to
157. Since the study was conducted from January to May 2021, this indicator corresponds to similar studies performed using the DSRM methodology in the same time frame [41-43]. As presented in Table 1, since 2006 the number of configurative articles has considerably exceeded the number of aggregate articles on this subject.
The advanced growth of configurative articles can be explained by the expansion of the range of tasks with DSS use. As previously stated, configuration articles describe the circumstances and tasks necessary for the successful deployment of a particular artifact. The more use cases appear, the greater the number of published articles. The study's results were obtained through a full-text analysis of articles from the produced corpus. Selected articles are presented in Table 2. These articles were selected from the general corpus based on such criteria as the representativeness of the sample for each of the time intervals to demonstrate consistent interest in this subject and the use of descriptions of real-world situations in the articles that could be re-analyzed by an independent researcher while achieving the same results. In other words, works ensuring reproducibility of the result were selected. Simultaneously, in the Artifact column of Table 2, the characteristic Type 1 refers to general support for decision-making with an explicit indication of the possibility of multi-criteria selection, whereas Type 2 refers to general support for decision-making without an explicit indication of the possibility of multi-criteria selection.
As indicated by the tables presented, the analysis of the corpus of articles confirms the hypothesis about the importance and unresolved issues associated with developing distributed neural-network DSS in various subject areas. The following sections of this article are devoted to a detailed analysis of the identified construction heuristics and the formulation of
key topics of our own research. According to the DSRM methodology, the standard table structure should include a column containing comparison metrics with analogues proposed by the authors of the works. However, this column is missing in Table 2, since none of the examined articles present any clearly defined criteria for comparing different approaches to the development of distributed neural-network DSS. This complicates the process of objectively analyzing current solutions and selecting close analogues. Thus, the development and validation of objective criteria for comparing neural network DSS can be considered as a promising research topic already at the stage of preliminary literature analysis.
3. Analysis results
3.1. Application of the neurosymbolic paradigm to the development of modern DSS
As previously stated, two fundamental questions must be addressed while developing the IDSS. The first question concerns the presentation of information and knowledge in such a manner that it is possible to use this knowledge in both symbolic and sub-symbolic systems. The second question is devoted to the expression of symbolic reasoning at the sub-symbolic level. Without resolving these issues, it is impossible to create neurosymbolic systems.
Given that every neurosymbolic system is composed of symbolic and sub-symbolic modules, the issue of communication between them plays a crucial part. In other words, it requires the capacity to convert symbolic knowledge to distributed knowledge and vice versa. This two-way translation is justified by the fact that ANNs cannot function at high levels of abstraction: reasoning at these levels can be only symbolic. Additionally, the question about the location of this encoded knowledge
Table 2.
The final analytical table. Selected works dedicated to the creation of the IDSS
Problem Artifact Construction heuristics Expectation heuristics Source
Pregnancy diagnosis assessment Type 1 Fuzzy logic 2-tuple Presence of expert assessments [44]
Supplier selection Type 1 Fuzzy logic Presence of historical data and criteria [45]
Scholarship winner selection Type 1 Fuzzy logic. Rule-based reasoning Presence of student applications [46]
Disease risk assessment Type 2 Rule-based reasoning. Genetic algorithms Presence of expert assessments and historical data [47]
Recommendations to clients Type 1 Fuzzy logic Presence of historical data [48]
Air cargo palletizing scheme Type 2 Rule-based reasoning. Genetic algorithms Presence of restrictions [49]
Selection of the most important risk for mitigation Type 1 Fuzzy logic 2-tuple. Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) Presence of expert assessments and alternatives [50]
Estimation of the water flow velocity Type 2 ANN Presence of historical data [51]
Flood risk assessment Type 2 Hierarchy analysis method Presence of expert assessments [52]
Selection of a model for transferring computing to cloud infrastructure Type 1 Fuzzy logic 2-tuple. Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) Presence of expert assessments and alternatives [53]
Accident prevention Type 2 Fuzzy logic.ANN Presence of historical data [54]
Supplier search Type 1 Fuzzy logic 2-tuple Presence of expert assessments and alternatives [55]
Best ERP software search Type 1 ANN.Hierarchy analysis method Presence of expert assessments and alternatives [56]
Best research project selection Type 1 Trust functions. The Dempster-Shafer theory Presence of expert assessments and alternatives [57]
Analysis of claims Type 2 Rule-based reasoning. Example-based reasoning Presence of a database of rules and a database of examples [58]
Analysisof the possibilities of using renewable energy sources Type 1 Rule-based reasoning Presence of digital cards [59]
in sub-symbolic modules arises: the knowledge may be stored in the ANN's weights, in the ANN's loss function, or in a representation in the form of factor models [60]. The selection of a method for representing knowledge in a way comprehensible for a neural network constitutes a part of the current agenda for this area of knowledge for the next decade [18]. Along with the technical issues of developing distributed representations, there are other important issues regarding the expression of connections between components of the encoded symbolic structure, as well as the encoding of first-order logic expressions [61].
There are various approaches to developing distributed representations of symbolic structures. One of these approaches is previously mentioned Tensor Representations [30]. This method makes it possible to create distributed representations of rather complex and recursive symbolic structures, for example, binary trees. A distinguishing characteristic of this approach in comparison to, for instance, the VSA (Vector Symbolic Architectures) paradigm, is the ability to decode a structure from a distributed representation without losses. The second critical feature is the ability to express a number of symbolic operations in the form of tensor manipulations.
This allows Tensor Representations to become the basis for non-symbolic algorithms, since a compiled ANN can serve as a universal performer of tensor operations. It is critical to note that developing neural networks that do not require training yet can execute complicated tasks is a challenging and promising issue [62, 63]. Thus, neurosymbolic systems based on Tensor Representations make it possible to express a certain symbolic algorithm in the form of compiled networks that do not require training, and are categorized as systems of the fifth type according to the classification described above. However, the
Tensor Representations are characterized by explosive growth as the depth of the encoded structure increases. Therefore, there are other approaches for encoding symbolic structures into a fixed-size vector. These approaches are referred to as VSA/HD (Vector Symbolic Architectures / High Dimensional Computing). The fixed length of the resulting vector is another disadvantage of such approaches: it is possible that the vector's dimensionality is insufficient to store the necessary information to reconstruct the symbolic structure.
Tensor Representations may also be utilized to address the second question that arises while developing neurosymbolic systems: the expression of symbolic reasoning at the sub-symbolic level. It has been demonstrated that simple algorithms (such as arithmetic operations) can be represented as a series of compiled neural networks [64, 65]. On the other hand, there are other approaches that allow us to express complex symbolic algorithms in the form of neural network architectures. Such architectures include, for example, Neural Turing Machines (NTN). They were first proposed in 2014 [66] and have since become widely used in various tasks, ranging from basic algorithmic tasks to reinforcement learning [67], sequential recommendations [68], natural language transduction [69], etc.
In neurosymbolic DSS, training plays a critical role since these systems must provide solutions, including those based on previous historical data (for example, setting weights for criteria or expert assessments based on prior decision-making examples). Furthermore, the results of sub-symbolic reasoning should be retrieved from the sub-symbolic system (ANN). On the one hand, this makes it possible to produce a completely trainable system; on the other hand, it allows the integration of symbolic reasoning and progression to a higher level of abstraction at any moment.
This once again raises the question of the interpretability of the results obtained via the use of neurosymbolic systems.
In neurosymbolic systems, it is necessary to take into account various requirements, such as modularity, the ability to work with a symbolic language, the ability to express reasoning and fulfill constraints at both symbolic and sub-symbolic levels. The problems of unified knowledge and reasoning expression stay pertinent for the development of neurosym-bolic DSS.
3.2. Possible implementation process
In multi-criteria decision-making tasks, the knowledge of the problem situation is hierarchical [26, 50, 53, 70] and may be represented as a tree, with leaves indicating linguistic evaluations and nodes representing corresponding elements such as criteria, sets of criteria, experts, and alternatives. The selection of linguistic evaluations is influenced by the need to account for context ambiguity, the unstructured character of expert assessments, and other factors. Thus, by encoding the knowledge of the problem situation using a tree, it is possible to express it in a distributed form without losing information, provided that Tensor Representations are utilized. According to the aforementioned properties of Tensor Representations, it becomes possible to extract required symbolic structures from a distributed representation and express a particular symbolic algorithm at the sub-symbolic level. The use of distributed representations and reasoning at the sub-symbolic level allows us to represent certain components of the DSS in the form of highly efficient distributed modules.
The encoding of the tree, as well as the extraction of structural elements, can be
performed in the form of compiled ANN constructed in accordance with the rules of Tensor Representations. In terms of the neurosymbolic paradigm, the step of aggregation of assessments is an example of reasoning. The representation of reasoning at the sub-symbolic level is indeed an urgent task. We propose to perform the aggregation step, which is considered as reasoning in neurosymbolic DSS, with the help of a trainable distributed neural-network architecture, such as Neural Turing Machines. As previously mentioned, while using this architecture, it is feasible to describe a mathematical algorithm for finding a weighted sum and several others. As a result, it becomes possible to develop a fully trainable neural network-based aggregator of linguistic evaluations that over time adapts to incoming data and uses historical data. In particular, such an algorithm could independently assign weights of experts' assessments and criteria (as well as sets of criteria) according to a multi-level methodology of linguistic decision-making [70]. In this case, the result of the network operation becomes an instance of Tensor Representations, which turns into a linguistic assessment after decoding. Thus, one of the most critical requirements for the DSS, the inter-pretability of the results, is met.
The development of neurosymbolic DSS is a promising scientific and practical task. The current set of technological solutions makes it possible to develop both symbolic and sub-symbolic modules for neurosym-bolic DSS characterized by fault tolerance, distribution, and high computational efficiency, as well as a high level of interpreta-bility, which is critical for a decision-maker while making strategic decisions.
Conclusion
Within the framework of this research, new directions for decision support system development were identified and analyzed using the DSRM methodology. The original hypothesis about a significant increase of the importance of distributed DSS in the context of global digitalization and the creation of new organizational forms was validated by the collection and analysis of a large number of relevant scholarly sources. A consistent tendency towards developing such systems utilizing integrated neural-network approaches was discovered. The most significant conclusions include the following statements:
♦ currently, ANNs are most often used in the DSS as a module for solving prediction and classification problems;
♦ numerous applications use distributed representations based on Tensor Representations due to the ability of expressing symbols
and symbolic operations at the tensor level and extracting symbols from such representations without losing information.
Simultaneously, the analysis indicates that the most promising areas of fundamental and applied research are the following:
♦ implementation of explanatory abilities for distributed DSS;
♦ modifications to the basic models of Tensor Representations for large-dimension cases;
♦ integration of prospective neural network architectures, such as Neural Turing Machines, into the DSS;
♦ development of the nomenclature of configurations and possible options for the implementation of neurosymbolic DSS on the basis of the research presented in this article. ■
Acknowledgments
The reported study was funded by RFBR, project number 19-37-90058.
References
1. Panetto H., Iung B., Ivanov D., Weichhart G., Wang X. (2019) Challenges for the cyber-physical manufacturing enterprises of the future. Annual Reviews in Control, vol. 47, pp. 200—213.
DOI: 10.1016/j.arcontrol.2019.02.002.
2. Moreira M.E. (2017). The agile enterprise: Building and running agile organizations. New York: Apress. DOI: 10.1007/978-1-4842-2391-8.
3. MacDonald T.J., Allen D.W.E., Potts J. (2016) Blockchains and the boundaries of self-organized economies: Predictions for the future of banking. Banking beyond banks and money. New economic window. Cham: Springer, pp. 279-296. DOI: 10.1007/978-3-319-42448-4_14.
4. Hu P., Dhelim S., Ning H., Qiu T. (2017) Survey on fog computing: Architecture, key technologies, applications and open issues. Journal of Network and Computer Applications, vol. 98, pp. 27-42. DOI: 10.1016/j.jnca.2017.09.002.
5. Ohsawa S., Akuzawa K., Matsushima T., Bezerra G., Iwasawa Y., Kajino H., Takenaka S., Matsuo Y. (2018) Neuron as an agent. Proceedings of the 6th International Conference on Learning Representations (ICLR 2018), Vancouver, Canada, 30 April — 3 May 2018. Available at: https://openreview.net/forum?id=BkfEzz-0- (accessed 5 July 2021).
6. Fitzgerald S. (2018) Design and implementation of a distributed neural network platform utilising crowdsource processing (MAI Thesis). Dublin: Trinity College Dublin.
7. Arcand J.-F., Pelletier S.-J. (1994) ADN-analysis and development of distributed neural networks for intelligent applications. Proceedings of the 1994 IEEE International Conference on Neural Networks (ICNN'94). Orlando, USA, 27-29 June 1994, vol. 3, pp. 1519-1524. DOI: 10.1109/ICNN.1994.374513.
8. James M., Hoang D. (1992) Design of low-cost, real-time simulation systems for large neural networks. Journal of Parallel and Distributed Computing, vol. 14, no 3, pp. 221-235.
DOI: 10.1016/0743-7315(92)90065-U.
9. Pi Y., Yuan Q., Meng X. (2012) A flexible implementation method of distributed ANN. Proceedings of the 2012 International Conference on Brain Inspired Cognitive Systems (BICS 2012). Shenyang, China, 11-14 July 2012, pp. 345-350. DOI: 10.1007/978-3-642-31561-9_39.
10. Meng X., Yuan Q., Pi Y., Wang J. (2007) A novel method of constructing ANN. Proceedings of the
2007International Symposium on Neural Networks (ISNN 2007). Nanjing, China, 3-7 June 2007, pp. 493-499. DOI: 10.1007/978-3-540-72393-6_59.
11. Khouzam B. (2014) Neural networks as cellular computing models for temporal sequence processing (PhD Thesis). Supelec.
12. Murshed M., Murphy C., Hou D., Khan N., Ananthanarayanan G., Hussain F. (2019) Machine learning at the network edge: A survey. Working paper arXiv:1908.00080.
13. Ghosh J., Hwang K. (1989) Mapping neural networks onto message-passing multicomputers. Journal of Parallel and Distributed Computing, vol. 6, no 2, pp. 291-330. DOI: 10.1016/0743-7315(89)90063-4.
14. Suresh S., Omkar S., Mani V. (2005) Parallel implementation of back-propagation algorithm in networks of workstations. IEEE Transactions on Parallel and Distributed Systems, vol. 16, no 1, pp. 24-34. DOI: 10.1109/TPDS.2005.11.
15. Tollenaere T., Orban G. (1991) Simulating modular neural networks on message-passing multiprocessors. Parallel Computing, vol. 17, no 4-5, pp. 361-379. DOI: 10.1016/S0167-8191(05)80142-5.
16. Yousefpour A., Nguyen B.Q., Devic S., Wang G., Kreidieh A., Lobel H., Bayen A., Jue J. (2020) Failout: Achieving failure-resilient inference in distributed neural networks. Working paper arXiv:2002.07386.
17. Garcez A., Lamb L. (2020) Neurosymbolic AI: The 3rd wave. Working paper arXiv:2012.05876.
18. Peffers K., Tuunanen T., Rothenberger M.A., Chatterjee S. (2007) A design science research methodology for information systems research. Journal of Management Information Systems, vol. 24, no 3, pp. 45-77. DOI: 10.2753/MIS0742-1222240302.
19. Pletnyakov V.A. (2012) The development of modern technologies to support management decision-making in strategic innovation. Terra Economicus, vol. 10, no 2.2, pp. 56-59 (in Russian).
20. Selyutina L.G. (2020) Development of modern technologies to support management decision-making in the service economy business. Proceedings of the International Scientific and Practical Conference
"Economics of XXIcentury ", Novosibirsk, 23 October 2020, pp. 387-393 (in Russian).
21. Power D.J. (2001) Supporting decision-makers: An expanded framework. Informing Science, vol. 1, pp. 1901-1915. DOI: 10.28945/2384.
22. Power D.J. (2008) Understanding data-driven decision support systems. Information Systems Management, vol. 25, no 2, pp. 149-154. DOI: 10.1080/10580530801941124.
23. Chernikov A.D., Eremin N.A., Stolyarov V.E., Sboev A.G., Semenova-Chashchina O.K., Fitsner L.K. (2020) Application of artificial intelligence methods for identifying and predicting complications in the construction of oil and gas wells: Problems and solutions. Georesources, vol. 22, no 3, pp. 87-96 (in Russian). DOI: 10.18599/grs.2020.3.87-96.
24. Bondarenko I.S. (2021) The concept of the simulation model of the decision support system of the coal enterprise. Vestnik Universiteta, no 2, pp. 42-50. DOI: 10.26425/1816-4277-2021-2-42-50 (in Russian).
25. Kureichik V.M. (2012) Features of decision making support system design. Izvestiya SFedU: Engineering Sciences, vol. 7, pp. 92-98 (in Russian).
26. Kuziemski M., Misuraca G. (2020) AI governance in the public sector: Three tales from the frontiers of automated decision-making in democratic settings. Telecommunications Policy, vol. 44, no 6, article ID 101976. DOI: 10.1016/j.telpol.2020.101976.
27. Yeung K. (2020) Recommendation of the council on artificial intelligence (OECD). International Legal Materials, vol. 59, no 1, pp. 27-34. DOI: 10.1017/ilm.2020.5.
28. Kelley T.D. (2003) Symbolic and sub-symbolic representations in computational models of human cognition: What can be learned from biology? Theory & Psychology, vol. 13, no 6, pp. 847-860. DOI: 10.1177/0959354303136005.
29. Pylyshyn Z.W. (1989) Computing in cognitive science. Foundations of cognitive science. Cambridge, MA: MIT Press, pp. 51-91.
30. Smolensky P., Legendre G. (2006) The harmonic mind: From neural computation to optimality-theoretic grammar. Vol. 1: Cognitive architecture. Cambridge, MA: MIT Press.
31. Plate T.A. (1995) Holographic reduced representations. IEEE Transactions on Neural Networks, vol. 6, no 3, pp. 623-641. DOI: 10.1109/72.377968.
32. Kanerva P. (2009) Hyperdimensional computing: An introduction to computing in distributed representation with high-dimensional random vectors. Cognitive Computation, no 1, pp. 139-159. DOI: 10.1007/s12559-009-9009-8.
33. Schlegel K., Neubert P., Protzel P. (2020) A comparison of vector symbolic architectures. Working paper arXiv:2001.11797.
34. Kautz H. (2020) The third AI summer: AAAI2020 Robert S. Engelmore memorial award lecture. Available at: https://youtu.be/_cQITY0SPiw (accessed 5 July 2021).
35. Mao J., Gan C., Kohli P., Tenenbaum J.B., Wu J. (2019) The neuro-symbolic concept learner: Interpreting scenes, words, and sentences from natural supervision. Working paper arXiv:1904.12584.
36. Johannesson P., Perjons E. (2014) An introduction to design science. Cham: Springer. DOI: 10.1007/978-3-319-10632-8.
37. Höckmayr B.S. (2019) Engineering service systems in the digital age. Wiesbaden: Springer Gabler. DOI: 10.1007/978-3-658-26203-7.
38. vom Brocke J., Havner A., Maedche A. (2020) Design science research: cases. Cham: Springer. DOI: 10.1007/978-3-030-46781-4.
39. Dresch A., Lacetda D.P., Antunes J.A.V. Jr. (2014) Design science research. A methodfor science and technology advancement. Cham: Springer. DOI: 10.1007/978-3-319-07374-3.
40. Alismail S., Zhang H., Chatterjee S. (2017) A framework for identifying design science research objectives for building and evaluating IT artifacts. Proceedings of the International Conference on Design Science Research in Information System and Technology. Karlsruhe, Germany, 30 May — 1 June 2017, pp. 218-230. DOI: 10.1007/978-3-319-59144-5_13.
41. Sutton R.T., Pincock D., Baumgart D.C., Sadowski D.C., Fedorak R.N., Kroeker, K.I. (2020) An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digital Medicine, no 3, article ID 17. DOI: 10.1038/s41746-020-0221-y.
42. Kasie F.M., Bright G., Walker A. (2017) Decision support systems in manufacturing: a survey and future trends. Journal ofModelling in Management, vol. 12, no 3, pp. 432-454. DOI: 10.1108/JM2-02-2016-0015.
43. Zhai Z., Martinez J.F., Beltran V., Martinez N.L. (2020) Decision support systems for agriculture 4.0: Survey and challenges. Computers and Electronics in Agriculture, vol. 170, article ID 105256. DOI: 10.1016/j.compag.2020.105256.
44. Song K., Zeng X., Zhang Y., Jonckheere J., Yuan X., Koehl L. (2021) An interpretable knowledge-based decision support system and its applications in pregnancy diagnosis. Knowledge-Based
Systems, vol. 221, article ID 106835. DOI: 10.1016/j.knosys.2021.106835.
45. Alavi B., Tavana M., Mina H. (2021) A dynamic decision support system for sustainable supplier selection in circular economy. Sustainable Production and Consumption, vol. 27, pp. 905-920. DOI: 10.1016/j.spc.2021.02.015.
46. Sukenda S.T.M.T., Puspitarani Y., Wahyu A.P., Yustim B., Sunjana (2021) Fuzzy logic implementation using the Tsukamoto method as a decision support system in scholarship acceptance. Turkish Journal of Computer and Mathematics Education (TURCOMAT), vol. 12, no 11, pp. 1411-1417. DOI: 0.17762/turcomat.v12i11.6054.
47. Aggarwal L., Goswami P., Sachdeva S. (2021) Multi-criterion intelligent decision support system for COVID-19. Applied Soft Computing, vol. 101, article ID 107056. DOI: 10.1016/j.asoc.2020.107056.
48. Zong K., Yuan Y., Montenegro-Marin C.E, Kadry S.N. (2021) Or-based intelligent decision support system for e-commerce. Journal of Theoretical and Applied Electronic Commerce Research, vol. 16, no 4, pp. 1150-1164. DOI: 10.3390/jtaer16040065.
49. Lee N.-S., Mazur P., Bittner M., Schoder D. (2021) An intelligent decision-support system for air cargo palletizing. Proceedings of the 54th Hawaii International Conference on System Sciences. Maui, Hawaii, 5-8 January 2021, p. 1405. DOI: 10.24251/HICSS.2021.170.
50. Labella A., Koasidis K., Nikas A., Arsenopoulos A., Doukas H. (2020) APOLLO: A fuzzy multi-criteria group decision-making tool in support of climate policy. International Journal of Computational Intelligence Systems, vol. 13, no 1, pp. 1539-1553. DOI: 10.2991/ijcis.d.200924.002.
51. Simsek S., Genc O., Albizri A., Dinc S., Gonen B. (2020) Artificial neural network incorporated decision support tool for point velocity prediction. Journal of Business Analytics, vol. 3, no 1, pp. 67-78. DOI: 10.1080/2573234X.2020.1751569.
52. Souissi D., Zouhri L., Hammami S, Msaddek M.H., Zghibi A., Dlala M. (2020) GIS-based MCDM-AHP modeling for flood susceptibility mapping of arid areas, southeastern Tunisia. Geocarto International, vol. 35, no 9, pp. 991-1017. DOI: 10.1080/10106049.2019.1566405.
53. Sohaib O., Naderpour M., Hussain W., Martinez L. (2019) Cloud computing model selection for e-commerce enterprises using a new 2-tuple fuzzy linguistic decision-making method. Computers & Industrial Engineering, vol. 132, pp. 47-58. DOI: 10.1016/j.cie.2019.04.020.
54. Dwi Putra M.S., Andryana S., Fauziah, Gunaryati A. (2018) Fuzzy analytical hierarchy process method to determine the quality of gemstones. Advances in Fuzzy Systems, vol. 2018, article ID 9094380. DOI: 10.1155/2018/9094380.
55. Jimenez Moya G.E., Zulueta Veliz Y. (2017) A 2-tuple linguistic multi-period decision making approach for dynamic green supplier selection. DYNA, vol. 84, no 202, pp. 199-206. DOI: 10.15446/dyna.v84n202.58032.
56. Yazgan H.R., Boran S., Goztepe K. (2009) An ERP software selection process with using artificial neural network based on analytic network process approach. Expert Systems with Applications, vol. 36, no 5, pp. 9214-9222. DOI: 10.1016/j.eswa.2008.12.022.
57. Weidong Z., Shiping G. (2009) Intelligent decision support system and its application in science research project selection. Proceedings of the 2009 First International Workshop on Education Technology and Computer Science. NW Washington, USA, 7-8March 2009, vol. 1, pp. 858-862. DOI: 10.1109/ETCS.2009.194.
58. Pal K., Palmer O. (2000) A decision-support system for business acquisitions. Decision Support Systems, vol. 27, no 4, pp. 411-429. DOI: 10.1016/S0167-9236(99)00083-4.
59. Voivontas D., Assimacopoulos D., Mourelatos A., Corominas J. (1998) Evaluation of renewable energy potential using a GIS decision support system. Renewable Energy, vol. 13, no 3, pp. 333-344. DOI: 10.1016/S0960-1481(98)00006-8.
60. Bengio Y., Deleu T., Rahaman N., Ke R., Lachapelle S., Bilaniuk O., Goyal A., Pal C. (2019)
A meta-transfer objective for learning to disentangle causal mechanisms. Working paper arXiv:1901.10912.
61. Bader S., Hitzler P., Holldobler S., Witzel A. (2007) A fully connectionist model generator for covered first-order logic programs. Proceedings of the Twentieth International Joint Conference on Artificial Intelligence (IJCAI-2007). Hyderabad, India, 6-12 January 2007, pp. 666-671.
62. Pinkas G. (1995) Reasoning, nonmonotonicity and learning in connectionist networks that capture propositional knowledge. Artificial Intelligence, vol. 77, no 2, pp. 203—247.
DOI: 10.1016/0004-3702(94)00032-V.
63. Pinkas G., Lima P., Cohen S. (2013) Representing, binding, retrieving and unifying relational knowledge using pools of neural binders. Biologically Inspired Cognitive Architectures, vol. 6, pp. 87-95. DOI: 10.1016/j.bica.2013.07.005.
64. Demidovskij A.V., Babkin E.A. (2020) Towards designing linguistic assessments aggregation as a distributed neuroalgorithm. Proceedings of the 2020 XXIIIInternational Conference on Soft Computing and Measurements (SCM). St. Petersburg, Russia, 27—29 May 2020, pp. 161-164. DOI: 10.1109/SCM50615.2020.9198767.
65. Demidovskij A., Babkin E. (2020) Designing arithmetic neural primitive for sub-symbolic aggregation of linguistic assessments. Journal of Physics: Conference Series, vol. 1680, no 1, article ID 012007. DOI: 10.1088/1742-6596/1680/1/012007.
66. Graves A., Wayne G., Reynolds M., Harley T., Danihelka I., Grabska-BarwMska A., Colmenarejo S.G., Grefenstette E., Ramalho T., Agapiou J., Badia A.P., Hermann K.M., Zwols Y., Ostrovski G., Cain A., King H., Summerfield C., Blunsom P., Kavukcuoglu K., Hassabis D. (2016) Hybrid computing using a neural network with dynamic external memory. Nature, no 538, pp. 471-476. DOI: 10.1038/nature20101.
67. Zaremba W., Sutskever I. (2015) Reinforcement learning neural Turing machines-revised. Working paper arXiv:1505.00521.
68. Chen X., Xu H., Zhang Y., Tang J., Cao Y., Qin Z., Zha H. (2018) Sequential recommendation with user memory networks. Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining (WSDM18), New York, 5-9February 2018, pp. 108-116. DOI: 10.1145/3159652.3159668.
69. Grefenstette E., Hermann K. M., Suleyman M., Blunsom P. (2015) Learning to transduce with unbounded memory. Working paper arXiv:1506.02516.
70. Demidovskij A.V., Babkin E.A. (2019) Developing a distributed linguistic decision making system. Business Informatics, vol. 13, no 1, pp. 18-32. DOI: 10.17323/1998-0663.2019.1.18.32.
About the authors
Alexander V. Demidovskij
Doctoral Student, Department of Information Systems and Technologies, National Research University Higher School of Economics, 25/12, Bolshaya Pecherskaya Street, Nizhny Novgorod 603155, Russia;
E-mail: ademidovskij@hse.ru
ORCID: 0000-0003-3605-6332
Eduard A. Babkin
Cand. Sci. (Tech.), PhD (Computer Science);
Professor, Department of Information Systems and Technologies, National Research University Higher School of Economics, 25/12, Bolshaya Pecherskaya Street, Nizhny Novgorod 603155, Russia;
E-mail: eababkin@hse.ru
ORCID: 0000-0003-2597-9043
