CC BY
27
DOI: 10.15514/ISPRAS-2023-35(1)-1
Design of an adaptable dashboard for smart cities
V. Contreras-Figueroa, ORCID: 0000-0003-2650-9748 <zs20000684@uv.mx> L.G. Montané-Jiménez, ORCID: 0000-0003-2732-5430 <lmontane@uv.mx> M. Cepero-García, ORCID: 0000-0002-0255-4256 <marite_cepero@live.com.mx> E. Benítez-Guerrero, ORCID: 0000-0002-5386-107X <edbenitez@uv.mx> C. Mezura-Godoy, ORCID: 0000-0001-5844-4198 <cmezura@uv.mx>
University of Veracruz, Xalapa, Veracruz, 91020, Mexico
Abstract. Today there are smart cities that, through the use of information technologies, sensors, and specialized infrastructure, focus their efforts on improving the quality of life of their inhabitants. From these efforts arose the need to analyze and represent data within a system to make it useful and understandable to people, for which dashboards emerge. The objective of these systems is to provide users with information to support decision-making, so it is essential to adapt the visualization of the information provided to their needs and preferences. However, the analysis of adaptability through user interaction and its benefits is a topic still under exploration. This paper analyzes the literature on information visualization in adaptable dashboards for smart cities. Based on the elements of adaptable dashboards identified in the literature review, we propose an adaptable dashboard architecture, identify the main characteristics of the users of a smart city dashboard, and build an adaptable dashboard prototype using user-centered techniques.
Keywords: Dashboard; Information visualization; Smart Cities; Adaptable system
For citation: Contreras-Figueroa V., Montané-Jiménez L.G., Cepero-García T., Benítez-Guerrero E., Mezura-Godoy C. Design of an adaptable dashboard for smart cities. Trudy ISP RAN/Proc. ISP RAS, vol. 35, issue 1, 2023, pp. 7-24. DOI: 10.15514/ISPRAS-2023-35(1)-1
Разработка адаптируемой информационной панели для умных городов
В. Контрерас-Фигероа, ORCID: 0000-0003-2650-9748 <zs20000684@uv.mx> Л.Г. Монтане-Хименес, ORCID: 0000-0003-2732-5430 <lmontane@uv.mx> М. Сеперо-Гарсия, ORCID: 0000-0002-0255-4256 <marite_cepero@live.com.mx> Э. Бенитес-Герреро, ORCID: 0000-0002-5386-107X <edbenitez@uv.mx> К. Мезура-Годой, ORCID: 0000-0001-5844-4198 <cmezura@uv.mx> Университет Веракруса, 91020, Мексика, Веракрус, Халапа
Аннотация. Сегодня существуют умные города, в которых за счет использования информационных технологий, датчиков и специализированной инфраструктуры повышается качество жизни жителей. При этом возникла потребность в анализе и представлении данных в некоторой системе, чтобы сделать их полезными и понятными для людей, для чего применяются информационные панели. Целью этих систем является предоставление пользователям информации для поддержки принятия решений, поэтому важно адаптировать визуализацию предоставляемой информации к их потребностям и предпочтениям. Однако анализ возможностей и преимуществ адаптивности посредством взаимодействия с пользователями - это тема, находящаяся на стадии изучения. В данной статье анализируется литература по визуализации информации в адаптируемых информационных панелях для
умных городов. На основе элементов адаптируемых информационных панелей, выявленных в обзоре литературы, мы предлагаем архитектуру адаптируемой информационной панели, определяем основные характеристики пользователей информационной панели умного города и создаем прототип адаптируемой информационной панели с использованием методов, ориентированных на пользователей.
Ключевые слова: информационная панель; визуализация информации; умные города; адаптируемая система
Для цитирования: Контрерас-Фигероа В., Монтане-Хименес Л.Г., Сеперо-Гарсия М., Бенитес-Герреро Э., Мезура-Годой К. Разработка адаптируемой информационной панели для умных городов. Труды ИСП РАН, том 35, вып. 1, 2023 г., стр. 7-24. DOI: 10.15514/ISPRAS-2023-35(1)-1
1. Introduction
Some cities are now called smart because of their ability to use information and communication technologies (ICT) to improve the quality of life of citizens and the overall operation of the city [1, 2]. The smart city approach incorporates ICT in almost all aspects of daily life in an urban space. Some areas that can take advantage of the integration of services and platforms are economy, government, environment, housing, society, and mobility, to mention a few [3]. A key smart city technology is the dashboard. According to Few [4], dashboards are a visual display of the most important information to support users to achieve one or more objectives in their daily lives.
Visualizations in dashboards show in a visual way - through charts and maps - the information of key performance indicators (KPIs). To be useful for decision-making, the visualizations of these KPIs must be carefully designed and then selected for deployment [5].
To meet the demand for information, it is important to consider the characteristics of the users. New forms of interaction can improve existing systems and create new platforms that contribute to the development of different areas of a city. Some recent research on the development of smart city-centric systems shows the importance of integrating the user into the development and deployment process of these systems [6-8].
In the current development of dashboards for smart cities there is a tendency to focus on a single user or a group with specific characteristics [9], which affects the deployment in cases where a user who is outside the specific context seeks to use the dashboard for their benefit. Some authors [1012] have explored the different information needs of dashboard users and tried to approach the problem from a user-centric perspective, which has led to the development of different dashboards for different users.
Sharifiq [13], through an analysis and description of information visualization using a flexible dashboard, shows how users can create their own configurations focused on what they want to visualize. This approach is a first attempt to make the interface adaptable to the user. This research analyzes adaptable dashboards in the context of smart cities, identifies city dashboards' users, and proposes an adaptable dashboard and its architecture. This paper is an extension of the paper originally presented at the 9th International Conference in Software Engineering Research and Innovation (CONISOFT 2021) [14]. The paper is organized as follows. Section 2 presents the methodology followed in carrying out the systematic review. Section 3 presents the quantitative and qualitative results of the review. Section 4 shows the proposed component specification as a result of the previous work. Section 5 shows the development of the interface and the user model. Section 6 shows the process carried out in evaluating the adaptable dashboard prototype. Finally, Section 7 discusses conclusions and future work.
2. Research process
We have conducted a systematic literature review following the methodology for systematic reviews developed by Kitchenham [15]. The research questions addressed by this study are: 8
Q1. How is key performance indicator information represented within the visual components of a dashboard?
Q2. What are the methodologies currently applied in the construction of dashboards? Q3. What are the benefits of adaptability applied to dashboards? Q4. What elements can be adapted within a smart city dashboard?
Based on the research questions and considering other terms obtained from previous research [1618, 9], we identified the following key words that are consistent with what is proposed to be found: "Smart city", "Adaptable dashboard", "Information visuali*ation", and "Key performance indicator". Once the terms had been defined, they were used to create the search string (S1):
S1. ("Smart city" OR "adaptable dashboard") AND ("Information visuali*ation" OR "keyperformance indicator").
To carry out an orderly review and to find valuable results for the research questions, we used the following selection criteria with which the works will have to comply to be considered:
• Inclusion:
o A research carried out in retrospective of no more than five years. o A research carried out in the area of computer science.
• Exclusion:
o Technical and programmatic platform development work. o Papers published in a language other than English. In order to search the resources to answer the research questions, we used the queries in four research databases: ACM Digital Library, IEEE Xplore, ScienceDirect, and SpringerLink. To extract the relevant papers for the systematic literature review, we followed a four-stage procedure:
1) Application of inclusion and exclusion criteria within search engines.
2) Synthesis of initial results and organization of metadata.
3) The initial selection process of articles through their title and keywords.
4) Selection process of papers by analysis of their abstract and contributions.
Fig. 1. Quantitative analysis of adaptability in dashboards
3. Review results
3.1 Quantitative analysis
The results of the search in the databases gave total of 542 articles, of which 518 were excluded, and 24 were selected for analysis. Based on the 24 selected articles, we performed a quantitative analysis of the problems and contributions identified in each paper through a bubble chart. The quantitative analysis of the contributions on adaptability in dashboards for smart cities is presented in Fig. 1. In this analysis, we identified that the greatest concentration of work is on how information is visualized on a dashboard.
3.2 Qualitative analysis
Within the visualization field, dashboards are commonly applied to monitor what is happening in a specific context so that people can interpret the visualization results and relate them to particular goals [19].
Tong and Wu [20] summarize the six characteristics of city dashboards: recording, connectivity, sensing, interaction, adaptation, and integration. Recording refers to saving city data. Adaptation refers to the ability to customize data products and services based on needs. Decision-makers can contrast the data through visual components (either static or real-time KPIs) to make comparisons and inferences to improve city operations [21] [22].
One of the biggest challenges of dashboards for smart cities is to satisfy the different information visualization needs of users to ensure that they are useful in their decision making [9]. In the current development of dashboards for smart cities, the aim is to provide the user with visual representations of data that are part of several screens [23].
To learn how to integrate adaptability into dashboards, we will analyze the literature in the following subsections. Each of the subsections corresponds to the answers to the research questions posed in Section 2.
3.2.1 Data visualization for city dashboard
City dashboards collect data from the urban environment for its analysis and display. Today there are multiple schemes (tools, frameworks, indices, indicators, and rankings) of urban data formed with a hierarchical structure of urban data analysis, where each level is described by the results of the previous level [24], [25].
Zdraveski [12] proposes a model with three scales of resolution or level of detail of the indicators: temporal (annual, quarterly, monthly, weekly, daily, hourly, and real-time values); spatial (values based on city, district, street, or GPS location); and human or population (values based on the city, region, municipality, neighborhood, household, or person).
The dashboards technology tries to solve the information overload for users by using visual components such as charts and tables to effectively communicate the city's current state and historical data to help identify patterns.
A recent survey [26] mentioned that it is vital for users of dashboards and smart city systems to manipulate the information for their benefit. According to this study, there are three main types of users: citizens, authorities, and communities.
A starting point to integrate the user in the information visualization process is to analyze how the user will obtain information from the system. There are several ways to represent and classify the information, Protopsaltis [19], and Peddoju [27] made a description of the most used charts in information visualization based on the analysis and pattern of data of interest. Their work concentrates on the main charts used when making decisions with a dashboard, as defined in Table 1, with univariate, bivariate, or multivariate variables. The charts are currently used in dashboards to represent multiple data and provide additional information that allows users to interpret the data. 10
Table. 1 Graphics used for the construction of dashboards
Data type Name Characteristics
Univariate Columns and bars [27] [19] Measurement of a variable according to a metric.
Pie [27] Illustrate the proportion of the elements that make up a whole.
Area [27] [19] To identify patterns between measurements of the variable and make comparisons.
Bivariate Scatter plot [27] Visualization of information in 2 and 3D for multidimensional analysis.
Heat map [27] [19] To show the spatial distribution of the variable on a map.
Line [27] [19] To explain functional dependencies between variables.
Multivariate Circular area (radar) [19] Comparison of multiple variables and their behavior among them
Stacked bars [19] Composition of data and categories that change over time.
Bubbles [27] [19] Specify large dimensions of variables within the same graphic.
Timelines [19] To understand the evolution of variables in relation to time.
Maps are another dashboard visualization component that presents geographic information through a digital representation of space, displaying location-related information [28, 29].
In addition to traditional charts and maps, specific visualization methods have been developed for certain scenarios. Purahoo [30] used a way of representing the information employing a gauge chart (speedometer-like chart) to represent the decibel level of the environmental sounds. Moustaka et al. [31] proposed a display to show the relationships between the various dimensions of a smart city through a model based on DNA structure. Another form in which the information can be represented is a 3D model [32].
The use of different ways of representing information allows developers to adapt these components according to what a user needs to achieve their goals. This representation also brings us to the challenge of building a dashboard with adaptable features that consider the user to select relevant information for their goals when viewing the dashboard and which tools it will display to promote a good experience.
3.2.2 Construction of dashboards
The construction of the scorecard of the city of Trieste in Italy developed by Brunetto [33], detailed analysis of its users and context. The dashboard was built considering the characteristics of the people who use it and the impact it would have within the government in which they work. Habibzadeh et al. [34] showed the characteristics for the construction of a smart city system in detail. As a starting point, [34] considers seeing these systems as a set of five different planes: I) of application, II) detection, III) communication, IV) data, and V) security. The model, despite being robust, does not consider the user in the construction.
One way in which user characteristics are considered to improve a system is the process applied by [28]. By using a three-dimensional design of the city and through data analysis with the Internet of Things (IoT) data, these systems provide crucial information to smart city dashboard end-users. Vicuna [29] applied a visualization process focused on the improvement of transportation within the city with transportation performance metrics (TPM), which consisted of applying steps for the analysis of information and the visualization process.
Rolim et al. [35] proposed an architecture used to build dashboards focused on how the user can comfortably interact with them. They use two architectures; the first is focused on constructing a dashboard for the visualization of information. The second is a dashboard for developers where they can modify and control the information on the visualization. Related to this, Chrysantina and Ivar
[36] in their work on user-focused dashboard design, mentioned some of the main issues to consider when developing a user-focused (or user-designed) dashboard.
An important contribution identified from the review of the literature was the Dashboard Design Guide developed by Few [16], in which he identified and described analogies to common problems in dashboards. Few also present guidelines for positioning and arranging elements to avoid overloading the user. The authors [36], [37] and [9], developed their research based on the characteristics of this guide.
Regarding the construction of adaptable dashboards, it was identified in the works of Ergasheva et al. [9], and Elshehaly et al. [37] similar steps in the process of building adaptable dashboards, so the following procedure is proposed for their construction, it can be seen in the Fig. 2.
Define The lisl Use a dashboard Selecting the visual Selecting the Improve the of potential KPJS design guide component ior each implementation dashboard
data type technology
Fig. 2. Procedure for dashboard construction
A fundamental part of applying methodologies to the development of dashboards is the possibility of validation through testing. Dostal et al. [38] developed a method of evaluation for smart city environments through models based on current and future information.
The literature review commonly mentions the process of building dashboards based on their technology and the steps to achieve a final product that meets the characteristics of information visualization. However, this only provides applied processes and patterns that, even though they follow a standardization, are not studied with the end users.
3.2.3 Adaptability in dashboards
One feature of dashboard design that has been little explored and implemented is adaptability. As mentioned in [9], the focus of current dashboards is a single person or a group of people who will be using the system. Making it clear the need to incorporate adaptability into the dashboard design to meet users' needs, and in turn, improve their decision-making based on the KPIs displayed on their screen. For example, Ergasheva et al. [9] built its InnoMetrics platform focused on energy efficiency, which adapts what is displayed on the screen according to the importance of an event that requires a solution.
The most crucial feature identified in the literature review is integrating user needs and preferences into the development of platforms related to information visualization, particularly dashboards for smart city. It must meet their expectations and objectives to be helpful as a tool to improve their quality of life and their interaction with their environment [39].
As mentioned by Han et al. [22], the creation of dashboards through user assistance tools to personalize the components which will display all the information requires a classification of the information to be subsequently displayed in a useful way to the user in its context. An efficient dashboard is one that considers the needs of users to ensure that the viewing process is completed smoothly [40].
Elshehaly et al. [37] developed a dashboard that actively involves the user during the whole process. Since the tool allows the user to adapt the information displayed within the cards, users feel comfortable visualizing what is necessary and can expand the information communicated through their interaction with the system.
The use of Nielsen heuristics [41] to develop a dashboard type system makes it possible to add features that make it attractive for users in their context of use. Within smart cities, understanding the needs of both the city and the users who are going to make decisions is crucial. One of the primary benefits of adaptability in dashboard is to improve the quality of the information to make decisions in a smart city. Information visualization helps to provide the information and facilitates the work of local authorities and people interested in monitoring the city. As a result of using visualization to achieve this, it is possible to build better opportunities for citizens and visitors [42].
3.2.4 Adaptable dashboard elements
The first type of adaptability identified in the literature analyzed was the operation of a visualization system. An example of this type of adaptability is shown in the document of Chan et al. [43], where graphical elements display information from sensors in an organized manner, first using a data analysis algorithm that provides an open standard for users to create compelling visualizations. The creation of dashboards through user assistance tools to customize the components that will display all the information requires a classification of the information to later display it in a useful way to the user in context [22]. Smart city dashboards function as a constant monitoring system that shows users information about their environment to help them make decisions [44]. According to Alves et al. [45], one way to adapt visual information is with zooming in and out on different sets of time-based details, which allows the user to specify the period in which they want to review the information.
In contrast, a particular way identified for adapting dashboards according to users and the context of smart cities is the use of Domain Specific Language (DSL), which focuses on using specific words and phrases with a syntax to make changes to the content displayed [46]. Silva et al. [47] identified that the number of KPIs used for smart city systems is relevant to modify the way in which information is displayed in a logical and orderly manner, thus improving the understanding of the information for the user. The organization of the KPIs information within a smart city system is an important feature that can be adaptable [47]. Silva et al. [47] proposed a structure that organizes the system elements involved in the information organization, from the document's structure. Following this same idea, Limon et al. [42], through an analysis based on the construction of smart cities, identified the critical parts when designing a platform focused on information visualization. This model seeks to improve the construction of web platforms by using a software engineering model [48]. Other factors identified previously in the literature as relevant components of an adaptable dashboard are: dashboard design guidelines [16] and the importance of integrating users and their information within the process [9] [35] [34]. Based on identified adaptable dashboard elements, we present the specification of components and an architectural proposal for an adaptable dashboard in the following section.
4. Structure of adaptable dashboard
Adaptable dashboards can be customized to present data according to the user knowledge to tailor what is being communicated. Therefore, adaptability characteristics in dashboards can enhance the user's data understanding and use of available information [18]. Based on the literature review and the model of [42], we identified elements that had not been previously considered such as design guidelines, user information for the construction of the dashboard, and a personalized selection of key performance indicators.
Smart city features are another component that is considered when designing adaptable dashboards. This information can be collected through sensors and databases. Subsequently, the selection of the KPIs used to display the information through visual components is made. We suggest building the visual components using the recommendations of the dashboard design guide developed by Few
[16]. And finally, to build the dashboard, we will use user preferences and interaction to adapt the dashboard interface.
We develop a functional architecture to show the internal structure of an adaptable dashboard (see Fig 3). Its parts will be described in the following sections to understand the adaptable dashboard proposal.
Fig. 3. Adaptable dashboard architecture
4.1 Users
The dashboard users can interact with the visualizations to select, filter, and adjust the components to change their shape and size. In this type of platform, it is necessary to consider the number of components and avoid information overload.
The number of components, the amount of information, and the way in which data are represented are elements that users unconsciously use to understand and use the dashboard visualizations [23]. Ivanov [49] mention the priorities of the users organized in the following way: performance monitoring, planning, communication, and consistency of data management. The authors Ivanov [49] and Young [50] define three main types of users for the dashboards, 1) Operational, 2) Analysts or managers and 3) Experts or executives. The three types of users analyzed will be considered for the dashboard.
Each dashboard user must use the information with the minimum effort to make a quick decision; thus, this information must be eye-catching and interactive to be meaningful and of constant use [49].
4.2 Smart city data
The dashboard for smart city is constructed in a way that its internal elements can constantly operate through data from sensors and databases. Some existing public databases pertain to specific countries or cities; however, these data are only available to the citizens through files or application programming interfaces [51, 52]. For example, in Mexico, an information transparency portal has been in place since 2018 that provides data files that are constantly updated to collect and display information for systems that support the work of the government [53].
The source of the data is relevant because it defines the way in which it will be used in the construction of visual representations. Current web platforms use sockets to transmit information in real time, while some others use databases that are constantly updated from automated systems. For the adaptable dashboard prototype, the Thingsboard [54] platform was selected, this platform consists of a web-based system focused on the control and deployment of dashboards for smart cities.
4.3 Adaptation
A system is called adaptable if it provides the user with tools that allow the user to change the characteristics of the system. Its objective is to provide the user with facilities to adapt the system to his personal tasks and needs. Control of the adaptation is given to the user, who must initiate the adaptation and use it [55].
Adaptable dashboards enable the selection of the most appropriate metrics and use them in a structured way. In this way, by adapting the interface to the user interaction, relevant information for each user is displayed when needed [56]. These adaptability features are implemented in a general dashboard interface to transform the way data is displayed and improve users' decision-making.
Familiarity can be generated in the user, reducing information overload, and allowing them to interact in a precise and easy way. Among the adaptable components that will be considered are: 1) Customization, 2) Zoom, 3) Colors, sizes and positions of the components, 4) Emphasize relevant information and 5) Make components simultaneously comparable [23].
Adaptability in dashboards enhances the display of information with an interactive mechanism that allows users to select the information to be displayed on their interface. In the dashboards already generated, adaptability allows the user to specify the information and the way it is presented within the graphical interface.
5. Adaptable dashboard development
As a starting point for designing adaptability within the dashboard, we considered the contributions of Strugar [56] and Young [50]. In their work, they mention that there is a set of key questions to consider in the design of a dashboard:
1) What metrics does the user need to visualize?
2) What context does each metric require to be meaningful?
3) Which visual representation best communicate s the metric?
The answers to the guiding questions were defined to have a first draft of what the system and the organization of the information within the dashboard deployment will be.
The first thing to do is to install the Thingsboard platform in a controlled local environment to define the features used for its creation and deployment on a personal computer, as shown in Table 2.
Table 2. Implementation technologies
Technology Version
Operating system Linux Ubuntu 20.04.3
Data base PostgresSQL 12.9
Java JDK OpenJDK 11.0.13
Thingsboard Realease 3.3.2
The details for the construction of the dashboard will be defined in the following sections. Each section answers the guiding questions posed above, the selected metrics, the type of user considered, the visual representation of the metrics, and the context to be used to classify the indicators employed.
5.1 Metrics for the user
Being a dashboard for smart cities, the metrics have a common context which is focused on improving processes and services within the city, however, the context of each of them changes according to what is required to show the user specifically. The scope of application of metrics according to the case study of the research project is focused on smart cities [57]. According to the classifications of Alvarado [58], Borjaz [2], Estrada [59], Bosh [60], Cohen[61] and Sharifi [25] the
following categories are defined for the development of the adaptable dashboard: 1) Economy, 2) People, 3) Government, 4) Environment, 5) Living, 6) Mobility and 7) Data. To perform the analysis of the performance indicators and whether they will be useful for the users of the smart city dashboard, we used the user-centered design tool called personas. Fig. 4 shows the relevant user characteristics of this instrument: 1) Demographic data, 2) Technology-related data, 3) Personal data, 4) User's motivation.
Context
MTth tftv Kifwtruriut». wh ffi t«ni IN* - < !«" 1 " at (IM* fvotn tf>v cit> Thn prmn twt'wfrr* hetr » ttahbouxl ■I <jt heps to make Coy Jo<3*y ffeoswm rcganhnf i happtttin| 11 th* smjtt city.
Hum
Fig. 4. Instrument Personas
The second instrument used was the empathy mapping tool (see Fig. 5). This map makes it possible to identify some interactions and experiences that users have when using the dashboard. It consists of four segments in which data is aggregated regarding what they think and feel, hear, see, say and do, and, finally, what are their pains and needs.
SSB-S»i Mi
«tetoltliMl fl^
j-.—UF.,I . ---- ^
||---~
Fig. 5. Empathy map
5.2 Visual representation of metrics
The tool selected for the dashboard has a pre-established set of charts that can be used for the construction. Some of the charts mentioned in the section IV are mentioned here, and new charts belonging to the tool have been added 1) Last-value charts, 2) Time series, 3) Remote controls, 4) Alarms, 5) Maps with position indicators, 6) Gauge plots, 7) Digital calendars and 8) Customizable graphics.
5.3 Dashboard design
The adaptable dashboard interface design (see Fig. 6) follows the style of the thingsboard platform. It has a side menu where users can select the section of the system where they want to be located, it has a home section where the recently consulted information is shown, a section to visualize their configured dashboards, an information section where they can generate information reports, they can consult the information sources and the data with which the system is being fed, and it has a gallery of components to modify the parameters of the used visualization. In addition, there is a configuration section that allows you to change the way the graphical interface looks as well as colors and font sizes.
Fig. 6 Adaptable dashboard The first image (see Fig. 6a) shows a configuration screen where users can select the different areas of interest with which they will interact. The general categories are represented as larger bubbles, while some subcategories and derived indicators are the smaller bubbles. This screen will serve as a starting point for users to select information relevant to them, from which a dashboard specific to their selection will be built. This interface complies with Raymond's usability rules [62] by making the selection of items limited to a number and with a consistent response to the user's actions. Figure 6b corresponds to the main dashboard menu. In this screen the user can switch between the different sections of the platform: dashboards, information, sources, gallery, and configuration. In the internal screen of the dashboard, the user can interact with the charts to modify their size and position; each of the charts is inside its widget, which shows information related to the data. At the bottom is the mechanism to adapt the interface, which is a drop-down component to avoid influencing the interaction with the data.
Finally, the fourth image shows the mechanism whereby the user will be able to modify the information that is currently on the screen. The category selector allows the user to focus his attention on a single type of information and, at the same time, to display more related information without having to use a search engine.
6. Dashboard evaluation
The objective of the evaluation was to gather information on the interaction and user experience of the adaptable dashboard for smart cities through tests with users.
As part of the evaluation, the prototype shown in the previous section was used to obtain feedback on the design and use of the adaptable dashboard mechanism. Specifically, we sought to evaluate if the component that adapts the interface with the labels (see Fig. 6d) is perceived as comfortable and easy to use.
Since the perception of the information for each user may vary, we used questionnaires to gather user feedback on different aspects of the prototype, so that they could make comments that would
help improve the dashboard. We selected the post-task Single Ease Questionnaire (SEQ) [63] to obtain feedback on specific tasks with the user interface, it has 1 question for each activity carried out with a scale of 1 to 7, in addition, three questions were added to collect user context information. We additionally used the post-study System Usability Scale (SUS) [64] questionnaire to obtain user feedback on the use of the prototype, it has 10 questions with a scale of 1 to 5. Both questionnaires complete a total of 16 questions in total. These questionnaires were selected because they have been used to evaluate dashboard interfaces in different fields of study [65-68]. Sauro [69] defines the reliability of the questionnaires used. The tasks that users had to complete are:
1) Create a new dashboard;
2) Enter a dashboard to view your charts;
3) Configure transport and energy charts in the dashboard.
In total, the evaluations were estimated to last a total of 10 minutes for each user to provide sufficient time for them to comment on the design and experience of using the prototype.
6.1 Participants and process
Four people participated in the dashboard test. Each of them lives within a city in the process of developing to be smart (Mexico City) [1]. Because they are in different locations, remote testing was conducted via video calls.
Users were asked to connect through videoconferencing software in which they were able to share their camera and screen. The first step of the testing was to provide the user with the research context, explaining what a dashboard is and its application in multiple areas. Then, the user was explained the testing procedure.
Subsequently, the user was informed of the letter of consent in which he/she agrees to the recording of the session and is notified that the data will be used privately for research purposes. A total of 3 links were provided to the user, the first was a form in which the user agreed to participate in the tests. The second was the link to the SEQ and SUS questionnaires (divided by pages), and the third was the dashboard prototype accessible via the web.
The environment for the test was set up when the user logged into the video call. We started by turning on his camera, verifying that when sharing the screen, he could share only the dashboard window and that he could access the Internet to view the questionnaires and the dashboard. The test was controlled by segmenting the entire session into 15-minute slots, which allocated 10 minutes for testing and 5 minutes for collecting feedback and having a backup space for the next participant.
6.2 Evaluation results
The results of the SEQ questionnaire are shown in Table 3. The results obtained are divided by the average, median, success rate, error rate, and average score obtained in the SEQ questionnaires.
Table 3. SEQ questionnaire results
Activity Average Median Success rate Error rate AVG score
1 64.33 58 75% 15% 4.25
2 22.33 18.5 100% 0% 7
3 23.33 20 100% 0% 6.25
AVG 36.66 32.16 91.6% 8.4% 5.83
The first result of interest is the time it took the users to complete activity 1 (creating a new dashboard). Besides being the one that took the longest time (64.3 seconds), this activity is the only task with a 75% success rate because one user was unable to complete the activity. We can also see this effect reflected in the median column, which for activity 1 was 58 seconds. Regarding the rating of the activity, the average was considerably low with a total of 4.25 being the maximum 7. This
means, in conjunction with the time obtained, that the task was complex to understand and that users have difficulties in configuring a new dashboard from the main interface.
Even though task 1 had a low result, tasks 2 and 3 had 100% success rate, each of them with a similar average time (22.3 and 23.3 seconds). The most notable difference between these two tasks lies in the rating obtained, task 2 (entering a dashboard) obtained a rating of 7, indicating that it is a simple task to perform and that there was no problem at all, while task 3 (configuring charts in the dashboard) obtained a rating of 6.25, where users considered it a little more difficult to perform. Finally, the overall rating of the dashboard prototype was 5.83, which indicates that there are still features that can be improved to give users a better experience when using the platform, these features fall mostly on the method of creating a new dashboard and the label system that adapts the interface.
Moreover, Table 4 presents the SUS evaluation results divided by user. The table shows both the values of the total sums of each questionnaire and the score obtained on a scale from 0 to 100.
Table 4. SUS questionnaire results
User Sum of ratings SUS Rating
1 35 87.5
2 30 75
3 33 82.5
4 29 72.5
Overall rating 79.37
In the ratings of this instrument, the lowest rating corresponds to user 4 with a value of 72.5 followed by user 2 with 75. In contrast, users 1 and 3 rated the usability of the dashboard with a higher value (87.5 and 82.5). This indicates that the opinions of the different users can tell us which parts of the system meet their objectives and which parts need to be corrected.
Despite rating differences, the overall grade of 79.37 is in an acceptable range according to the interpretation of the results developed by [70], specifically, the value corresponds to a grade of 'C', where a value lower than 60 is an F, between 60 and 69 to D, between 70 and 79 to C, between 80 and 89 to B and higher than 90 is an A.
The result obtained, being right in the middle of the classification with the letter C, indicates that there are still factors that can be improved for users enjoy interacting with the system, making it useful and easy to use. In the answers of this questionnaire, we were able to analyze that both the way of creating dashboards and the functionality of adaptation by tags is functional to the user, but there are sections that, with the changes suggested by them, will improve the way in which the usability of the adaptable dashboard for smart cities is perceived.
Once the results of the questionnaires have been analyzed, it is important to mention the main comments made by users about the design and functionality of the prototype dashboard. Feedback from users is valuable because it will serve as a basis to understand what is happening with the dashboard and for further studies to improve the functionalities and aspects of the user interface.
7. Conclusions and future work
This paper presents the results of a systematic review of the literature on information visualization in adaptable dashboards. We used the methodology proposed by [15] for the elaboration of literature reviews, setting out a search strategy that starts with the research questions. We used a search string in digital databases to obtain a total of 24 relevant articles used to answer the research questions posed. In addition, using the information related to adaptability, a proposal for component specification was built for later use in the design of an adaptable dashboard. We used the user-centered tools personas and empathy map to know the characteristics of the user and then propose and evaluate the design of the interface that will allow adapting the dashboard for smart city. The evaluation was carried out with 4 users with a total of 3 tasks. It was carried out using 2 evaluation
19
instruments: SEQ and SUS. The results obtained for the SEQ questionnaire is a total of 5.83 points out of 7. The results of the SUS questionnaire gave a value of 79.37 being in an acceptable rating. As future work, user feedback must be addressed in both the design and development of the dashboard to improve the user experience. Furthermore, once a refined design has been obtained, it is necessary to put it into practice in a real environment.
References
[1] Alvarado-López R.A. Ciudades inteligentes y sostenibles: una medición a cinco ciudades de México. Estudios Sociales. Revista de Alimentación Contemporánea y Desarrollo Regional, vol. 30, issue 55, 2020, 28 p. (in Spanish).
[2] Borja Zapata J.S., Arce Ruiz R.M., Soria Lara J.A. Modelo para evaluar el cumplimiento de objetivos de ciudades inteligentes. In Proc. of the Congresso Internacional "Sustentabilidade Urbana", 2018, pp. 225234 (in Spanish).
[3] Tachtler F.M. Best way to go? Intriguing Citizens to investigate what is behind smart city technologies. In Proc. of the ACM Conference on Designing Interactive Systems, 2017, pp. 28-33.
[4] Few S. Dashboard Confusion Revisited. Perceptual Edge, 2007, pp. 1-6. URL: http://perceptualedge.com/articles/visual_business_intelligence/dboard_confusion_revisited.pdf.
[5] Vila R.A., Estevez E., Fillottrani P.R. The design and use of dashboards for driving decision-making in the public sector. In Proc. of the 11th International Conference on Theory and Practice of Electronic Governance, 2018, pp. 382-388.
[6] Latifah A., Supangkat S.H., Ramelan A. Smart Building: A Literature Review. In Proc. of the International Conference on ICT for Smart Society, 2020, pp. 1-6.
[7] Cepero-Garcia M.T., Montane-Jimenez L.G. Visualization to support decision-making in cities: Advances, technology, challenges, and opportunities. In Proc. of the 8th International Conference in Software Engineering Research and Innovation (CONISOFT), 2020, pp. 198-207.
[8] Laramee R.S., Turkay C., Joshi A. Visualization for Smart City Applications. IEEE Computer Graphics and Applications, vol. 38, issue 5, 2018, pp. 36-37.
[9] Ergasheva S., Ivanov V. et al. InnoMetrics Dashboard: The Design, and Implementation of the Adaptable Dashboard for Energy. In IFIP Advances in Information and Communication Technology, vol. 582, Springer, 2020, pp. 163-176.
[10] Tokola H., Groger et al. Designing manufacturing dashboards on the basis of a key performance indicator survey. Procedia CIRP, vol. 57, 2016, pp. 619-624.
[11] Pettit C., Widjaja I. et al. Visualisation support for exploring urban space and place. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. 1, 2012, pp. 153-158.
[12] Zdraveski V., Mishev K. et al. ISO-Standardized Smart City Platform Architecture and Dashboard. IEEE Pervasive Computing, vol. 16, issue 2, 2017, pp. 35-43.
[13] Shafiq S.I., Szczerbicki E., Sanin C. Proposition of the methodology for Data Acquisition, Analysis and Visualization in support of Industry 4.0. Procedia Computer Science, vol. 159, 2019, pp. 1976-1985.
[14] Contreras-Figueroa V., Montané-Jiménez L.G. et al. Information visualization in adaptable dashboards for smart cities: A systematic review. In Proc. of the 9th International Conference in Software Engineering Research and Innovation (CONISOFT), 2021, pp. 34-43.
[15] Kitchenham B. Procedures for performing systematic reviews. Keele University Technical Report TR/SE-040, NICTA Technical Report 0400011T.1, 2004, 34 p.
[16] Few S. Information Dashboard Design: Displaying Data for AT-A-Glance Monitoring. Analytics Press, 2013, 260 p.
[17] Zhang J., Johnson K.A. et al. Human-Centered Information Visualization. Proceedings of the International Workshop on Dynamic Visualizations and Learning, 2002, 7 p.
[18] Matheus R., Janssen M., Maheshwari D. Data science empowering the public: Data-driven dashboards for transparent and accountable decision-making in smart cities. Government Information Quarterly, vol. 37, issue 3, 2020, article no. 101284, 2020, 10 p.
[19] Protopsaltis A., Sarigiannidis P. et al. Data visualization in Internet of Things: Tools, methodologies, and challenges. In Proc. of the 15th International Conference on Availability, Reliability and Security, 2020, article no. 110, 11 p.
[20] Tong X., Wu Z. Study of Chinese city "portrait" based on data visualization: take city dashboard for example. Lecture Notes in Computer Science, vol. 10919, 2018, pp. 353-364.
[21] Batty M. A perspective on city dashboards. Regional Studies, Regional Science, vol. 2, issue 1, 2015, pp. 29-32.
[22] Han Q., Nesi P. et al. Smart City Dashboards: Design, Development, and Evaluation. In Proc. of the IEEE International Conference on Human-Machine Systems, 2020, 4 p.
[23] Dobraja I., Kraak M.-J., Engelhardt Y. Facilitating insights with a user adaptable dashboard, illustrated by airport connectivity data. Proceedings of the International Cartographic Association, vol. 1, 2018, article no. 30, 6 p.
[24] Kourtit K., Nijkamp P. Big data dashboards as smart decision support tools for i-cities. an experiment on Stockholm. Land Use Policy, vol. 71, 2018, pp. 24-35.
[25] Sharifi A. A typology of smart city assessment tools and indicator sets. Sustainable Cities and Society, vol. 53, 2019, article no. 101936, 15 p.
[26] O'Neill D., Peoples C. Using IT to monitor well-being and city experiences. IEEE Potentials, vol. 35, issue 6, 2016, pp. 29-34.
[27] Peddoju S.K., Upadhyay H. Evaluation of IoT data visualization tools and techniques. In Data Visualization, Springer, 2020, pp. 115-139.
[28] Lv Z., Li X. et al. Government affairs service platform for smart city. Future Generation Computer Systems, vol. 81, 2018, pp. 443-451.
[29] Vicuna P., Mudigonda S. et al. A generic and flexible geospatial data warehousing and analysis framework for transportation performance measurement in smart connected cities. Procedia Computer Science, vol. 155, 2019, pp. 226-233.
[30] Purahoo Z., Cheerkoot-Jalim S. SenseAPP: An IoTBased Mobile Crowdsensing Application for Smart Cities. In Proc. of the 3rd International Conference on Emerging Trends in Electrical, Electronic and Communications Engineering, 2020, pp. 47-52.
[31] Moustaka V., Maitis A. et al. CityDNA Dynamics: A Model for Smart City Maturity and Performance Benchmarking. In the Companion Proceedings of the Web Conference, 2020, pp. 829-833.
[32] Chen S., Dong H. Visualizing Toronto City Data with HoloLens: Using Augmented Reality for a City Model. IEEE Consumer Electronics Magazine, vol. 7, issue 3, 2018, pp. 73-80.
[33] Brunetto O. City dashboards: The case of Trieste: Trieste overview Lecture Notes in Computer Science, vol. 10407, 2017, pp. 710-721.
[34] Habibzadeh H., Kaptan C. et al. Smart City System Design. ACM Computing Surveys, vol. 52, issue 2, 2019, pp. 1-38.
[35] Rolim D., Silva J. et al. Web-based development and visualization dashboards for smart city applications. Lecture Notes in Computer Science, vol. 12128, 2020, pp. 337-344.
[36] Chrysantina A., Sœb0 J.I. Assessing user-designed dashboards: a case for developing data visualization competency. IFIP Advances in Information and Communication Technology, vol. 551, Springer, 2019, pp. 448-459.
[37] Elshehaly M., Randell R. et al. QualDash: Adaptable generation of visualisation dashboards for healthcare quality improvement. IEEE Transactions on Visualization and Computer Graphics, vol. 27, issue 2, 2021, pp. 689-699.
[38] Dostal R., Pribyl O., Svitek M. City infrastructure evaluation using urban simulation tools. In Proc. of the Smart Cities Symposium Prague, 2020, 6 p.
[39] Lu M., Liu S. et al. Design and Implementation of Power Information Visualization Platform Based on Smart Meter. In Proc. of the 3rd International Conference on Information Science and Control Engineering, 2016, pp. 297-301.
[40] Curry E., Fabritius W. et al. A Model for Internet of Things Enhanced User Experience in Smart Environments. In Real-time Linked Dataspaces, Springer, 2020, pp. 271-294.
[41] Molich R., Nielsen J. Improving a Human-Computer Dialogue. Communications of the ACM, vol. 33, issue 3, 1990, pp. 338-348.
[42] Limon-Ruiz M., Larios-Rosillo V.M. et al. User-oriented representation of Smart Cities indicators to support citizens governments decision-making processes. In Proc. of the 5th IEEE International Smart Cities Conference, 2019, pp. 396-401.
[43] Chan A.L., Chua G.G. et al. Practical experience with smart cities platform design. In Proc. of the IEEE World Forum on Internet of Things, 2018, pp. 470-475.
[44] Few S. Why Most Dashboards Fail. Perceptual Edge, 2007, 2 p. URL: https://www.perceptualedge.com/articles/misc/WhyMostDashboardsFail.pdf.
[45] Alves A.P., Milani A.M., Manssour I.H. Visual Analytics System for Energy Data in Smart Cities and Buildings. In Proc. of the IEEE International Smart Cities Conference, 2020, 8 p.
[46] Rojas E., Bastidas V., Cabrera C. Cities-Board: A Framework to Automate the Development of Smart Cities Dashboards. IEEE Internet of Things Journal, vol. 7, issue 10, 2020, pp. 10128-10136.
[47] Silva J., Mojica J. et al. Algorithms for the Control of Key Performance Indicators for Smart Cities. Procedia Computer Science, vol. 170, 2020, pp. 971-976.
[48] Pressman R. Ingeniería del software: Un enfoque practico. 7ma. McGrawhill, 2010, 777 p. (in Spanish).
[49] Ivanov V., Larionova D. et al. Design of a dashboard of software metrics for adaptable, energy efficient applications. In Proc. of the International Distributed Multimedia Systems Conference on Visualization and Visual Languages, 2019, pp. 75-82.
[50] Young G.W., Kitchin R., Naji J. Building city dashboards for different types of users. Journal of Urban Technology, vol. 28, issues 1-2, 2021, pp. 289-309.
[51] Open data, Paris. URL: https://opendata.paris.fr/pages/home/, 19 de enero 2022.
[52] VancouverCity. URL: https://vancouver.opendatasoft.com/pages/help/, 19 de enero 2022.
[53] Gobierno de México. URL: https://datos.cdmx.gob.mx/group/educacionciencia-y-tecnologia, 19 de enero 2022.
[54] Thingsboard. URL: https://thingsboard.io/, 19 de enero 2022.
[55] Oppermann R. Adaptively supported Adaptability. International Journal of Human-Computer Studies, vol. 40, issue 3, 1994, pp. 455-472.
[56] Strugar D. Complex Systems: On Design and Architecture of Adaptable Dashboards, Lecture Notes in Computer Science, vol. 11771, 2019, pp. 176-186.
[57] U4SSC - Collection Methodology for Key Performance Indicators for Smart Sustainable Cities, 2017, 134 p.
[58] Alvarado López R. Ciudad inteligente y sostenible: una estrategia de innovación inclusiva. PAAKAT: Revista de Tecnología y Sociedad, vol. 7, issue 13, 2017, 17 p. (in Spanish).
[59] Estrada E., Peña A. et al. Algoritmos para el control de indicadores clave de desempeño para Smart Cities. Research in Computing Science, vol. 147, issue 8, 2018, pp. 121-133 (in Spanish).
[60] Bosch P., Jongeneel S. et al. CITYkeys indicators for smart city projects and smart cities. European Commission within the H2020 Programme, Technical Report, 2017, 306 p.
[61] Cohen B. What exactly is a smart city? Co. Exist, issue 19, 2012. URL: https://www.fastcompany.com/1680538/what-exactly-is-a-smart-city, 19 de enero 2022.
[62] Raymond E.S., Landley R.W. The Art of UNIX Usability. 2004. URL: http://www.catb.org/esr/writings/taouu/taouu.html, 19 de enero 2022.
[63] Tedesco D., Tullis T. A comparison of methods for eliciting post-task subjective ratings in usability testing. In Proc. of the Usability Professionals Association Conference, 2006, pp. 1-9.
[64] Brooke J. SUS: A 'Quick and Dirty' Usability Scale. In Usability Evaluation in Industry, CRC Press, 1996, pp. 107-114.
[65] Yamamoto S., Mori H., eds. Human Interface and the Management of Information. Visual Information and Knowledge Management. Lecture Notes in Computer Science, vol. 11569, 2019, 652 p.
[66] Celi E. Application of Dashboards and Scorecards for Learning Models IT Risk Management: A User Experience. Lecture Notes in Computer Science, vol. 9188, 2015, pp 153-165.
[67] Wu D.T.Y., Vennemeyer S. et al. Usability testing of an interactive dashboard for surgical quality improvement in a large congenital heart center. Applied Clinical Informatics, vol. 10, issue 5, 2019, pp. 859-869.
[68] Ullmann T., De Liddo A., Bachler M. A visualization dashboard for contested collective intelligence learning analytics to improve sensemaking of group discussion. RIED. Revista Iberoamericana de Educaci'on a Distancia, vol. 22, issue 1, 2019, p. 41-80.
[69] Sauro J., Lewis J. Standardized usability questionnaires. In Quantifying the User Experience. 1st edition.
Morgan Kaufmann, 2012, pp. 185-240.
[70] Bangor A., Kortum P., Miller J. Determining What Individual SUS Scores Mean: Adding an Adjective Rating Scale. Journal of Usability Studies, vol. 4, issue 3, 2009, pp. 114-123.
Information about authors / Информация об авторах
Victor CONTRERAS-FIGUEROA - Master student in User-Centered Interactive Systems. Main research interests: Visualization, Human-Computer Interaction, Smart Cities and Web Development.
Виктор КОНТРЕРАС-ФИГЕРОА - студент магистратуры в области интерактивных систем, ориентированных на пользователя. Основные научные интересы: визуализация, взаимодействие человека с компьютером, умные города и веб-разработка.
Luis Gerardo MONTANÉ-JIMÉNEZ, Ph.D. in Computer Science, Professor at the Faculty of Statistics and Informatics. Main research interests: Computer-Supported Cooperative Work (CSCW), Visualization, Human-Computer Interaction, Context-Aware Computing and Videogame Development.
Луис Херардо МОНТАНЕ-ХИМЕНЕС, кандидат компьютерных наук, профессор факультета статистики и информатики. Основные исследовательские интересы: совместная работа с компьютерной поддержкой, визуализация, взаимодействие человека и компьютера, контекстно-зависимые вычисления и разработка видеоигр.
María CEPERO-GARCÍA - Master's Degree in User-Centered Interactive Systems, Professor at the Faculty of Statistics and Informatics of the Universidad Veracruzana. Her areas of interest are: Computer-Supported Cooperative Work (CSCW), Human-Computer Interaction, Data visualization, and Smart Cities.
Мария СЕПЕРО-ГАРСИА - профессор факультета статистики и информатики. Сферы ее интересов: совместная работа с компьютерной поддержкой, взаимодействие человека и компьютера, визуализация данных и умные города.
Edgard BENÍTEZ-GUERRERO - Ph. D. in Computer Science from the University of Grenoble in France, Professor at the Faculty of Statistics and Informatics. Research interests: Human Computer Interaction, Artificial Intelligence, Collaborative Computing, Data Management and Visualization. Эдгар БЕНИТЕС-ГЕРРЕРО - кандидат компьютерных наук Гренобльского университета во Франции, профессор факультета статистики и информатики. Научные интересы: взаимодействие человека с компьютером, искусственный интеллект, совместные вычисления, управление данными и визуализация.
Carmen MEZURA-GODOY - Ph. D. in Computer Science from the University of Savoie in France, Professor at the Faculty of Statistics and Informatics. Main research interests: Human Computer Interaction, User Experience, Visualization, Computer Support Collaborative Work and Multiagent Systems.
Кармен МЕЗУРА-ГОДОЙ - кандидат компьютерных наук Университета Савойи во Франции, профессор факультета статистики и информатики. Основные исследовательские интересы: взаимодействие человека с компьютером, пользовательский опыт, визуализация, совместная работа компьютерной поддержки и многоагентные системы.
