Repositioning People in Creative Futures: A Method to Create Sound Advice with Exploratory Scenarios Текст научной статьи по специальности «Экономика и бизнес»

Repositioning People in Creative Futures: A Method to Create Sound Advice with Exploratory Scenarios

Guillermo Velasco

Senior Researcher"; Honorary Research Fellowb; Expertc, guillermo.velasco@upm.es

Rafael Popper

Honorary Senior Lecturerb; Founder and Director'; Principal Scientist1; rafael.popper@manchester.ac.uk

Ian Miles

Emeritus Professor, Manchester Institute of Innovation Researchb; Academic Supervisor, Laboratory for Economics of Innovation, Centre for Science and Technology, Innovation and Information Policy, Institute for Statistical Studies and Economics of Knowledge (ISSEK)e,

i.d.miles@manchester.ac.uk

a Polytechnic University of Madrid, 28040 Madrid, Spain b Manchester Institute of Innovation Research, University of Manchester, Oxford Rd, Manchester M13 9PL, UK c Futures Diamond Ltd., Raise Chartered Accountants Trimble House, 9 Bold Street Warrington, WA1 1DN Manchester, UK d VTT Technical Research Centre of Finland Ltd., P.O. Box 1000, FI-02044 VTT, Finland e National Research University Higher School of Economics, 11, Myasnitskaya str., Moscow 101000, Russian Federation

Abstract

Foresight scenarios are not only useful presentational devices to show that many aspects of the future are open. Scenarios are means for generating advice that helps policymakers initiate actions in the present or near future that will be of long-term significance. Despite the influence that such advice may have on policy decisions, the Foresight literature has paid very little attention to the creation of policy recommendations. Though reports of scenario exercises frequently conclude with lists of recommendations that follow from the study, there is very little explication of the process whereby advice is elicited from the examination of these future scenarios. This paper addresses this gap, examining how the generation of recommendations is related to the development of scenarios within multiple

future repositioning workshop settings. It focuses on the fluency and originality of these recommendations, and how this is influenced by repositioning participants in highly transformational scenarios. Repositioning is the process whereby participants are invited to imagine themselves playing roles in hypothetical future contexts, and on that basis to make decisions or devise strategies as if they actually were immersed in these circumstances. The method proposed and the findings of the case study have implications for why and how this future repositioning approach can be incorporated as a 'key feature' in the design of Foresight activities. The aim is also to raise awareness of the need for more exploration of Foresight recommendation methodology.

Keywords: reposition; foresight; advice; recommendation; futures; innovation; scenario; European Research Area (ERA); soundness; fluency; originality; creativity

Citation: Velasco G., Popper R., Miles I. (2021) Repositioning People in Creative Futures: A Method to Create Sound Advice with Exploratory Scenarios. Foresight and STI Governance, 15(2), 25-38. DOI: 10.17323/2500-2597.2021.2.25.38

0 I © 2021 by the authors. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.Org/licenses/by/4.0/).

Foresight is widely recognized to be a policy instrument that provides structured anticipation through the examination of alternative futures [Dator, Rodgers, 1991; De Jouvenel, 1967; Gabina, 2005; Godet, 1992; Havas, 2005; Kuwahara, 1996; Malaska, 2001; Miles et al., 2008a,b]. In particular, Foresight activities are often seen as having two main contributions to policymaking [Georghiou et al., 2008]. First is Foresight's capacity to deliver policy advice: the activity may be designed to establish priorities or build roadmaps, to examine the robustness of policies across different scenarios, or to appraise the consequences of different courses of action within changing circumstances. A second feature that has also gathered attention is the use of Foresight to facilitate networking and knowledge transfer, to "join up the innovation system" or to align different stakeholders' understanding of emerging issues. Foresight thus not only supports policymaking by providing information drawn from a wide range of knowledge sources but can also strengthen policy implementation by facilitating policy action through learning processes and knowledge sharing across stakeholders [Da Costa et al., 2008; Eriksson, Weber, 2008; Popper et al., 2007; Salo, Cuhls, 2003]. Foresight activities can be seen as having anticipatory and recommending phases, the former consisting of explicating ongoing changes and alternative futures, the latter concerning the development of policy advice based on such understandings.

The policy dimension of Foresight may contribute to the integration of different policy actions, prioritizing S&T agendas, and even the creation of partnerships between public and private actors [Miles, 2008]. Many policy decisions made today are liable to have long-term implications for social, economic, and environmental affairs, and Foresight can help policymakers expand their time horizons beyond the short term. The usefulness of Foresight has been recognized by the European Commission; its systematic mapping of foresight initiatives in Europe and the world highlighted the systematic, participatory, long-term, and pragmatic character of the discipline [European Commission, 2002; Popper, 2009].

By questioning conventional assumptions about future prospects, Foresight facilitates a better understanding of plausible paths and 'visions of change' [Ramos, 2017]. The hope is that policies can be more precisely and effectively formulated in light of these future visions.1 The utilization of plausible scenarios also reveals the ethical dimension of Foresight [Bussey, 2014], since it invites policy designers to avoid concentrating their efforts solely on the most immediate present problems, and to consider the needs of future generations. Foresight, and the utilization of future scenarios in particular involves taking the future seriously as a 'principle of present ac-

tion' [Slaughter, 1995], since images of the future can shape the actions taken in the present. Foresight activities typically involve a combination of multiple techniques, some of which are more logical and deductive (e.g. data analytics, computer simulation), some of which involve more imagination and/or group discussion [Popper, 2008a, 2008b]. Various methods and rationales for the creation of images of the future and the articulation of alternative scenarios have been discussed in the literature, and several overviews exist [Carleton et al., 2015; Masini, 1982; Medina, 1999; UK Government, 2017). Foresight activities frequently include participative and interactive workshops as settings in which to develop alternative futures. Such workshops are defined as "temporary socio-spatial crystallisations of expertise, with a particular sort of socio-spatial group dynamics, in which different instruments and tools are deployed in order to endorse knowledge creation" [Duf-va, Ahlqvist, 2015]. Workshop participants deconstruct present narratives or contexts and create new empowering and plausible ones for themselves [Inayatullah, 2004]. By co-developing new narratives and visions, the participants develop a sense of engagement and ownership [Ramos, 2017].

It is crucial that Foresight results are found useful and participant stakeholders can feel themselves strengthened and empowered by effective recommendations [De Smedt, 2013]. However, the legitimacy of Foresight is sometimes questioned, not least because of the loose connections between the Foresight activity and the actual decision-making process [ Uotila et al., 2005]. This is not just a matter of policymakers necessarily having to take into account political machinations and democratic pressures alongside (and sometimes overruling) the recommendations stemming from long-term analyses. Often a long and complex road runs from the formulation of advice to its eventual acceptance and implementation. There is also the issue that these recommendations sometimes appear to emerge from a "black box". How the Foresight process has resulted in proposals for action remains largely obscure. Thus, documenting the way in which recommendations are related to the prior anticipatory phase - including the production of scenarios - should help secure them more legitimacy. Surprisingly, given that the creation of visions is crucial for the recommending phase of Foresight - during which a range of alternative actions or policy recommendations are generated - very few studies have documented the process whereby Foresight projects generate recommendations drawing on these visions.

Why has there been so little study of the recommending phase in the literature? Perhaps it is because decisions to implement recommendations, regardless of the process utilized to produce them, frequently de-

1 The term "vision" in English can refer to the capacity of sight and to an image of a possible state of affairs - which may have been produced by the capacity of foresight, but may also have connotations of something rather more supernatural, fantastical, or even psychopathological. Unfortunately, English words such as "imaginary" or "image" are also ambiguous terms. We use "vision" here to mean the more serious appraisal of a possible state of (future) affairs.

pend on a variety of unavoidable and uncontrollable external (often political) influences. There is a substantial body of work, deriving from Operational Research and related approaches, on ways in which choices may be made between alternative actions, they may be ranked in terms of priority, and so on. These methods include Multiple Criteria Analysis, Analytic Hierarchy Analysis, Action Roadmapping, among others [UK Government, 2009; Mardani et al., 2015, Popper et al., 2020]. These technical approaches aim at assessing costs and benefits of options in terms of various criteria; and even so political considerations may prevail when choices are actually made. While sophisticated tools may aid selection among various options, the question of how these options are arrived at receives much less attention (for discussion of morphological analysis, see [Alvarez, Ritchey, 2015]. Foresight practitioners have frequently paid more attention to the dynamic and creative processes developed during the definition of future scenarios than to the (potentially dynamic and creative) process of elaborating advice. The lack of explicit methodologies to create sound recommendations during the advice phase of a Foresight activity means that the suggestions that emerge from scenario studies frequently appear to be rather spontaneous and informal. This recommendation stage is often portrayed as simply involving participants proposing options for action and then engaging in some process of selecting among these. The approach to suggesting options may involve basic brainstorming, perhaps with some more structured elicitation of ideas as related to different policy actors and stakeholders. For example, in the "carousel" setting, sets of participants are asked to move around flip-charts representing different actor types; they annotate each chart with suggestions concerning actions and possibly other ideas, such as timetables for actions, indicators of successful implementation, and so on. [Miles et al., 2016]. In a similar vein, simulation gaming can be used to ask participants to assume different roles ("personas") of some particular actors in the scenarios and to discuss what their perspectives, objectives, and actions might be. Participants' creativity is expected to be higher where these are brought into "collision" - when participants have to think of alliances and counterstrategies. Selection between the ideas generated with these processes may involve, for example, an Eisenhower-matrix type mapping of their attractiveness and feasibility.2 Participants that have worked on different scenarios may have been asked to make proposals based on the issues highlighted by their own scenarios. There may be an effort at "windtunneling", that is, seeing how far particular policies remain valuable across different scenarios [Ringland, 2006]. But how are the ideas tied to the scenarios that have been used?

This is not the only topic that remains underexplored in the Foresight literature. There are very few systematic comparisons of different methods - probably because Foresight activities are rarely conducted as scientific experiments, but mainly as inputs to policy or strategy processes (one exception is the comparison of Delphi and cross-impact approaches to the same topic [Scapolo, Miles, 2006]). In the present context, how might scenario workshop methodology affect the ideas generated in those scenarios and the advice that is derived from them? The present paper represents a modest attempt to address these questions, by presenting a method for "repositioning" people in several future scenarios, and comparing the advice derived from immersion in those different contexts. It demonstrates that it is possible to examine such processes during the course of a policy-focused foresight activity. The hope is that a better understanding of the factors that affect the construction of policy advice in Foresight activities using scenario analysis can contribute to the development of more creative and effective recommendations emerging from the process, and that this in turn will increase the prospects for their actual implementation and for the long-term vision really being built into policymaking.

Types of Advice

Advice is a broad concept. It can refer to a single recommendation or compilation of such recommendations (what should or should not be done), and it can also involve detailed explanations of what logic underlies such recommendations (why it should or should not be done). There are the following classifications of advice [Dalal, Bonaccio, 2010]:

• Advice in favor of a specific alternative

• Advice against one or more alternatives

• Information: neutral advice providing information on alternatives, avoiding prioritizing or favoring any of them

• Decision support: provide support and guidelines on the decision-making process

Advice for a specific alternative has the capacity to summarize the problem into a precise solution, thus enabling faster decision-making processes [Schrah et al., 2006]; but this sort of advice may eventually limit the decision-maker's autonomy [Caplan, Samter, 1999; Goldsmith, 1994]. In extremely urgent situations, this may be a cost worth bearing, but such a restriction of freedom can lead to reactance on the part of decision-makers or to loss of self-esteem [Fisher et al., 1982]. There may be less of a sense of losing autonomy in relation to other types of advice that give higher levels of freedom to decision-makers, i.e., advice against alter-

2 This method is described in [Miles et al., 2016], but does not use this terminology; for an example see [Huang et al., 2016].

3 The full list of the recommendations derived from this process can be found the ERA Open Advice report [Popper et al., 2015a].

natives, information-oriented advice, or decision support advice.

Some experiments suggest that, in general, the type of advice most preferred by decision-makers is information-oriented advice [Dalal, Bonaccio, 2010]. However, this also depends on the geo-political implementation context since, as pointed out by [Keenan, Popper, 2008], in some regions (e.g., South America), there is a longstanding tendency to avoid openly making recommendations to the government due to the risk of appearing to be critical of current policy. Only when the advisor is seen as a credible expert, decision-makers may prefer prescriptions, recommendations in favor of specific alternatives. Interestingly, advice against alternatives was not found to be among the decision-makers' preferred types of advice. There are suggestions that information-oriented advice is more useful for newer rather than experienced decision-makers [Heath,Gonzalez, 1995]. Advisors should offer a specific type of advice for each contextual circumstance, e.g. on the different foresight scenarios, and try systematically to include sufficient information on the proposed alternatives [Dalal, Bonaccio, 2010]. The recommendations analyzed in the case study of this paper are broadly in line with Dalal and Bonac-cio's "Information: neutral advice providing information on alternatives" [Dalal, Bonaccio, 2010]. Furthermore, advice can play various roles, among which the category of "providing alternatives not considered by the decision maker" is probably the one most relevant to the present study and to Foresight activities in general. Other functions are the provision of emotional support, of arguments to endorse preconceived options, of insights into decision processes, and so on [Gibbons, 2003]. A slightly different classification of advice proposed by [Cross et al., 2001] features some categories overlapping with [Gibbons, 2003]. Some are functions of many Foresight activities - for instance, supplying sources of further information and proposing ways of reformulating the problem. These authors point out that different types of advice may be complementary, though it would usually be foregrounded.

Stimulating Creative Advice

As compared to basic methods of forecasting, such as trend extrapolation and simulation modeling, Foresight processes are intended to stimulate creative thinking [Staton, 2008] and enable collective learning [Harper, Pace, 2007]. The participants create a 'shared collage of futures', which is a valuable output of the workshop in its own right and one that supports the generation of actions. The question is raised of how far the generation of creative ideas with Foresight involves not just the imagination of people, but also results from the anticipation methodology employed [Dufva, Ahlqvist, 2015]. This is relevant to the case study discussed below.

From a broad perspective, triarchic theory [Sternberg, 1985] suggests that intelligence is composed of three parts or dimensions: a) a componential di-

mension related to the human capacity for analyzing problems (in the case of making recommendations in Foresight activities, this analysis often draws on future scenarios), b) an experiential one related to creativity and intuition (i.e. original ideas facilitate the selection of ways to solve problems that are not business-as-usual), and c) a practical dimension related to adaptation to the context (it sounds reasonable to believe that a high number of alternative ideas elicited in Foresight workshops would increase the chances that final recommendations generated with these ideas are compatible with the actual environment and circumstances). Creativity is not just evident during the anticipation phase (when scenarios are developed) but throughout the whole Foresight process - including the recommendation phase. The generation of recommendations itself involves a practical application of creativity, though it is quite possible that some recommendations are more or less closely modeled on ideas of which participants were already aware. Rietzschel et al. argue that for ideas to be creative, they need to be both original (unusual) and feasible (useful) [Rietzschel et al., 2010]. The fluency of ideas is seen to be a characteristic of creative people and is arguably as relevant in the recommendation phase as in the design of scenarios. In everyday use, "fluency" has connotations of the easy and flowing articulation of messages. Here we follow [Guilford, 1950, 1967] in using the term more restrictively simply to refer to the ability to produce numerous ideas. Eliciting a large number of ideas can enrich the Foresight process, since it allows for discussion around more action alternatives. Although policymakers themselves may not welcome a long list of options for action, the generation of numerous alternatives should increase the possibilities for selecting possible solutions to the problems they address. There is a huge amount of literature exploring the individual and social psychology of creativity [Sternberg, 1998, Glover et al., 1989, Martin, Wilson, 2018; Paulus, Nijstad, 2019; Dorfler, Stierand, 2020]. Although general aspects of creativity differ between individuals, there is a consensus in the literature that fluency and originality of ideas are distinct functions of the concept of creativity. These two elements, together with the flexibility and elaboration of ideas are usually used to measure the outcomes of divergent thinking processes [Guilford, 1950, 1967; Torrance, 1968, 1974; Amabile, 1983; Weisburg, 1986; Paulus, 2000; Kincaid, Duffus, 2004].

The case study described below will explore the capacity of future scenarios to increase the fluency and originality of individuals' ideas in Foresight recommendation processes.

Case Study Description and Rationale

This study draws on a set of workshops focusing on the future of the European Research Area (ERA). The European Commission's Framework Programme

(FP7) funded a Foresight project in 2012 on the future of the ERA by 2030 [Daimer et al., 2015]. The project, named "Forward Visions on the European Research Area" (VERA) was implemented from February 2012 to January 2015. It aimed to "provide relevant strategic intelligence for the future governance and priority-setting of the RTDI (Research, Technology Development and Innovation) system in Europe and for better adapting science, technology and innovation policy to the shifting global environment and upcoming socio-economic challenges."4 A key aspect of the project was the special attention paid to the actor's definition and selection, based on the stakeholder salience model.5

VERA established a set of four exploratory scenarios. These were built with a factor-oriented approach. Key factors were identified, alternatives projections developed for each factor, scenarios defined in terms of combinations of these alternatives, and these scenarios were elaborated upon as texts to be discussed later in the project. VERA differs from many other Foresight projects in that the scenario building team (and workshop participants) did not design or organize the recommendation phase of the project. Thus, participants in the recommending phase should not take part with a sense of "ownership" of, or commitment to, any particular scenario. Table 1 presents the resulting scenarios. While Scenarios 1 and 2 represented incremental changes in the governance of RTDI, the research landscape and socioeconomic context, Scenarios 3 and 4 reflected new socio-technical regimes, associated with transformative structural changes.6 Whereas decision-makers will often prefer to envisage futures that involve little change from those extrapolated within a "business as usual" scenario, foresight practitioners have long stressed the importance of challenging these cosy assumptions with scenarios that envisage more transformational change [Dator, 2009; Kahane, 2012]. While these exercises have often helped to provoke substantial change (Kahane [Kahane, 2012] in particular discusses the fall of apartheid in South Africa) and while there are ample examples of business-as-usual scenarios leading to failures to anticipate major shocks to the system, there has been little systematic compar-

ative analysis of the effects of employing scenarios of different types in Foresight projects. This essay is based upon an action research study that accompanied VERA for fifteen months, from January 2013 to March 2014 [Velasco, 2017]. So, instead of the Foresight process simply being conducted according to the facilitators' notions of good practice, an effort was made to document choices and actions in the exercise and to establish what the consequences of specific design decisions were.7

VERA is a major European Commission-funded project that applies Foresight to policy matters.8 VERA developed a communication flow across different ERA representative actors from multiple knowledge domains, regions, and functional levels. The project facilitated a strategic conversation between these actors to identify strategic options and recommendations around ERA with a long-term perspective. It is hoped that the findings could contribute to designing Foresight processes and methodological choices in future activities.

We cannot generalize from our results to say that these are the typical, let alone the only, pattern of outcomes that will characterize scenario-based Foresight activities, of course. Many more studies would be required to reach such a conclusion. We will look at VERA to identify patterns and phenomena that might be intelligible and explicable on the generation of sound advice. While these might not be replicated in other circumstances, the variations in patterns and themes across different future activities is something that can help us build a theory about the Foresight recommending processes. Research might be designed so as to explore this possibility; practice could be designed so as to capitalize upon it.

The analysis focused on the contribution of scenarios and actors to generate policy advice. As noted above, the connection between future scenarios and the generated advice is a black box, a gap in the Foresight literature. The study of VERA allowed for the possibility of opening the black box to start to bridge this gap. The VERA Foresight process is unusual, in being one in which the anticipatory and the recommending phases were transparently connected and documented.

4 http://eravisions.eu/, accessed 26.03.2021.

5 The Mitchell's stakeholder salience model [Mitchell et al., 1997] offers a political, operational, and dynamic approach to identifying stakeholders, taking into account the actors' legitimacy, power of negotiation, and perception of urgency. Delimitating the composition of stakeholder representation in collective thinking processes demands a meticulous identification of actors who are relevant to the process, as well as a clear design of the dynamics guiding their participation. Mitchell's salience model, which was also initially conceived for the business sector, has had interesting applications in policy intelligence [Haegeman et al., 2012].

6 As highlighted by [Popper et al., 2015b], "...there is not one clearly preferred scenario across all the focus groups. For each scenario we actually have stakeholder groups that do not find it desirable. Overall, the societal challenge scenario and the scenario with "experts at the wheel" to focus on sustainability are most often seen as desirable. Interesting deviation of that pattern can be seen, as representatives of the academic world (and to some extent by industry) see major disadvantages in a focus on "local solutions" and a shift in knowledge production towards a less science driven paradigm, as well as by societal actors who opposed the top-down definition of societal challenges. However, we clearly see that a VERA scenario dominated by private industry R&I is least desired across all stakeholder groups, even the majority of industry representatives did not find this scenario attractive."

7 As an action research, the key objectives of the project were to observe, explicate, criticize, and transform social practices. Potential objects of such inquiries include individuals, collectives, patterns, procedures, structures, or behaviors. Action research assumes that there are various ways of actively exploring such objects, interacting with people in the process and acknowledging the subjectivity associated to the researcher's observations [Ladkin, 2004]. Action research is a methodology that iteratively poses questions, plans actions, promotes reflection on research inquiries, seeks alternative actions and explanations, and monitors outcomes [McKernan, 1996] thus enabling open access to the project process and results.

8 In Yin's [Yin, 2014] terms, VERA could be considered a 'critical case'.

Table 1. VERA's Four Alternative Futures

VERA Scenarios Description

1. "Private KnowledgeGlobal Markets" Scenario 1 "assumes that today's European Research Area gradually evolves into what one might call a Global Innovation Area, where research is mainly legitimised by its contribution to innovativeness, competitiveness and growth. As a result of limited public funds, growing inequalities between Member States and the jostling for political influence within Europe, private actors, mainly firms, dominate the financing of the research landscape and thus the setting of research priorities."

2. "Societal Challenges-Joint Action" In Scenario 2, "today's European Research Area has developed its research and innovation capacities incrementally as efficient responses to the Grand Challenges. This means that economic growth and job creation have become challenges themselves, and that issues like climate change or health protection are perceived as Grand Challenges. In Europe as is the case globally, RTDI and education are considered key preconditions for the creation of sound solutions to these Grand Challenges."

3. "Solutions Apart -Local is Beautiful" Scenario 3 "captures the vision that today's concept of progress is transformed into a human-centred rationale, where e.g. happiness and quality of life are operationalised into new measures of progress. The after-effects ofthe global economic crisis are felt deep into the 2020s, and especially so in specific European Member States. Rather than driving societies and Member States apart, economic disparities in Europe create a new sense of community in the pursuit of well-being for all, including the RTDI system."

4. "Times of Crises -Experts at the Wheel" Scenario 4 assumes that "today's economic rationales (jobs and growth) have been transformed into an approach where a sustainable development path is viewed as the main rationale of progress. Human activities are limited by resource availability and the carrying capacities of ecosystems at all levels -ranging from local cultivation of land to the use of global commons such as the atmosphere. The sustainability rationale has therefore been adopted around the globe, but at different speeds and in a variety of ways".

Source: compiled by the authors using [Teufel et al., 2013].

Exploring the Process of Making Recommendations

The objective of the VERA recommending phase was "to underpin an adaptive, efficient, effective and well-resourced European Research Area (ERA) that fosters innovation and creativity and addresses upcoming socio-economic challenges by a) engaging with key stakeholders to explore strategic responses on the critical issues for the ERA evolution, and b) providing sound recommendations on research and innovation (R&I) policies and their governance and coordination across ERA" [Popper et al., 2015 a,b]. This objective is aligned with the VERA overall mission, which assumes that providing strategic intelligence to the governance of ERA requires the gathering of strategic ideas/responses from the participation of R&I key actors, while acknowledging that intelligent solutions to ERA challenges should be necessary, sufficient, and feasible enough to present adequate levels of soundness.

VERA's recommending phase mobilized a wide representation of European R&I stakeholders (73 participants). It involved seven different focus group workshops. Each of these focused on one part of the spectrum of ERA actors: civil society (Vienna), academia (Manchester), industry (Helsinki), research funders (Berlin), experts on ERA (Barcelona), policymakers (Barcelona), and international stakeholders (Brussels). The selected participants had to represent different knowledge domains and have not participated in the previous design workshops for the VERA scenarios. A 'literal replication' [Yin, 2014] was achieved through the replication of the approach across seven workshops: they shared the same coordinators, methodology, language (English), length, and presentation material, and took place in similar facilities. This increased the ca-

pacity to compare the outputs of different workshops on a level playing field.

Each workshop consisted of the following steps9:

1. Presentation of the four scenarios to the participants, by means of documents and a short video. Participants were then asked to individually vote on their most and least desired scenarios.

2. The three most desirable scenarios were used to constitute three discussion groups, with three to four stakeholder participants in each.

3. Each group could select a second scenario of their choice, but all groups were encouraged to consider discussing the least desirable scenario so as to make sure that all four alternative futures inspire or inform the recommendation process.

4. Groups undertook a conversation about two specific future scenarios, with a facilitator prompting the group to address opportunities, threats, and related recommendations.

Table 2 and Table 3 presents a description of the VERA workshops' activity. It shows the number of discussion groups and participants that talked about each scenario. We can see that scenario 2 was the one most discussed by participants, 44 persons, in contrast to scenario 3, which was debated by 30 participants. The average number of participants per group was very similar across scenarios.

Participants were thus asked about R&I system opportunities and perceived threats, as seen particularly from the standpoint of their own institutions, in the event of the scenario materializing. We describe this as them being asked to "reposition" themselves in the scenario, while retaining their institutional allegiances and interests as a stakeholder. Repositioning therefore

9 For more detailed reporting on the actual stakeholders' strategies and desirability of the scenarios see [Popper et al., 2015b; Velasco, 2017].

Table 2. Number of Discussion Groups in the VERA Focus Groups

Stakeholder groups Scenario

1 2 3 4

Society 1 2 1 2

Academia 2 2 1 1

Industry 1 1 1 3

Funders 1 3 1 1

ERA experts 1 2 2 1

International 1 1 1 1

Policymakers 3 1 1 1

TOTAL 10 12 8 10

Source: compiled by the authors.

refers to the process whereby participants situate their mindsets in a hypothetical future context and adopt decisions or devise strategies as if they were living or immersed in these contextual circumstances. All the opinions given by the group were collected in a flip chart and discussed (although agreement was reached by simple discussions, in some cases voting was needed) in order to achieve consensus. A high degree of consensus eventually improves the quality and accuracy of the collective advice, as suggested by [Ya-niv, 2004].

A large volume of insights was elicited from the seven workshops. systematic cleaning and filtering processes were implemented and applied separately for each workshop, so that we could differentiate and understand the position of each type of stakeholder. Every message generated in a focus group workshop was saved in a database and labeled according to the following criteria:

• In which scenarios was the insight generated?

• Which group discussion (and facilitator) generated the insight?

• Was the insight perceived by the group as an opportunity or as a threat?

• Did the opportunity or threat refer to the R&I system as requested, or instead can it be read just as a stakeholder's particular concern?

Table 3. Number of Participants in the VERA Focus Groups

Stakeholder groups Scenario

1 2 3 4

Society (9 persons) 3 6 3 6

Academia (12 persons) 8 8 4 4

Industry (10 persons) 4 3 3 10

Funders (11 persons) 4 11 4 3

ERA experts (13 persons) 4 9 9 4

International (6 persons) 3 3 3 3

Policymakers (12 persons) 12 4 4 4

Total participants for each scenario 38 44 30 34

Participants per group 3.80 3.66 3.75 3.40

Source: compiled by the authors.

• Was the message clearly a stakeholder's recommendation for policymakers or, instead can it be just read as a strategic action of that specific actor? Since the workshop methodology allowed two scenarios to be discussed by different facilitators, it is possible for different discussion groups to arrive at similar insights related to a given scenario. Such repeated insights were merged into one (with this merging process recorded). The merging process tried to maintain the participants' original expressions, while avoiding interpretations to preserve transparency. Quality of advice benefits from the integration of insights generated from multiple and non-correlated advisors [Soll, 1999; Johnson et al., 2001]. After this cleaning procedure, the original database was reduced by around 30% and every single message could be tracked to identify its generating discussion and corresponding facilitator. An overlapping analysis evaluated how many times the same insight was suggested in different scenarios. The coincidence of recommendations across different advising processes should increase decision-makers' confidence in these recommendations [Budescu et al., 2003].

A summary of the future repositioning and insights integration method can be found in Table 4. This method is recommended in exploratory-scenario-based foresight projects whose number of participants allows the organization at least four discussion groups. In practical terms, the method requires that facilitators be able to record every elicited insight, so that the analyst could afterwards track them and more efficiently undertake the processes of overlapping and insight integration. Identifying recurrent insights across multiple scenarios and discussion members provides the method with a way of eliciting broader and more diverse advice than would be generated by other recommendation-making alternatives.

Analysis of Results

As noted above, there is a lack of evidence concerning the influence that the anticipatory phase of Foresight has on the recommending phase. Future scenarios bring specific elements, contextual circumstances, values, and perspectives into Foresight discussions, and could stimulate participants' mindsets. This analysis of participants' reactions to the VERA scenarios allows us to examine whether scenarios have the capacity to stimulate the generation of insights. We shall here consider only spontaneous insights directly generated by participants during the workshops. In the final writing and argumentation phases of the Foresight process, advice was elaborated on the basis of the recommending phase, but this is not the focus of the present study. Two aspects of the insights generated in the VERA workshops will be studied: fluency (the number of insights generated per participant in each scenario) and the originality of these insights.

Table 4. Future Repositioning and Insights Integration Method

Repositioning • Present and allocate future scenarios to the discussion groups, making sure that all alternative futures presented inspire or inform the recommendation process • Ask participants about R&I system opportunities and perceived threats, as seen particularly from the standpoint of their own institutions, in the event of the scenario materializing (as if they were repositioned and immersed in that scenario) • Ask participants to formulate possible recommendations to address the opportunities and threats identified in the context of the scenario

Insights/ Integration • Label (and save) each elicited insight with information (an auxiliary worksheet is recommended) that includes the scenario upon which this particular insight was generated, and the group discussion/facilitator that generated it. • For each scenario, integrate all repeated insights (across different facilitator discussions) into one that faithfully captures the idea. Thus, you will obtain a shorter list of distinct insights per scenario. • Analyze how many times the same insight was suggested in different scenarios. The coincidence of a recommendation across different scenarios and groups increases decision-makers' confidence in that advice. A list of insights could be provided with a score or ranking based on these cross-scenario repetitions

Source: compiled by the authors.

Fluency Analysis

The generation of multiple ideas from participants in Foresight projects should in principle allow for the production of more practical solutions and recommendations. Table 5 shows the number of insights generated per participant of the different VERA focus groups for each scenario. While the data does not permit tests of statistical significance, some of these differences are quite striking - notably the differences between least and most stimulating in the society, academic, and policymaker groups.

The table shows that, in terms of insights produced per participant, Scenario 4 was the most stimulating context (see mark ++) for four actors (society, academia, experts on ERA-relevant initiatives, and international stakeholders). Scenario 3 was the most stimulating for two actors (industry and policymakers). Only research funders found Scenario 1 the most stimulating. Scenario 2 did not emerge as the most stimulating for any set of stakeholders.

Considering which scenarios prompted less fluency, Table 5 also shows that Scenario 3 was never the least stimulating scenario for any set of stakeholders (see mark --). Scenario 4 was the least stimulating future scenario for one group (research funders); Scenario 2 the least stimulating for two (academia and policymakers); and Scenario 1 was the least stimulating for four (society, industry, ERA experts, and international stakeholders).

Figure 1 represents the stimulation capacities of every scenario across the seven stakeholders, demonstrating that in the VERA process, Scenarios 4 and 3 had a higher capacity of insight stimulation than Scenarios 1 and 2, whose rate is similar.

Table 6 also adds some observations concerning the stimulation capacity of scenarios, presenting the data in terms of the ranking of scenarios. As well as presenting the number of times that each scenario emerged as the most, and as the least, stimulating, we present a score of each scenario in terms of "Ranking Points" (where least stimulating is ranked 1, most stimulating 4). Scenarios 1 and 2 were more stimulating than Scenarios 3 and 4 only once, while scenarios 3 and 4 were more stimulating than Scenarios 1 and 2 on six occa-

sions. Scenarios 3 and 4 were found least stimulating only once (42 in relation to 28 total points). This analysis invites reflection on the characteristics differentiating Scenarios 1 and 2 from Scenarios 3 and 4. As described above, the VERA scenarios involved two different types of transitions of the European research landscape. While Scenarios 1 and 2 represented an incremental evolution of RTDI governance, Scenarios 3 and 4 were the consequence of deep structural changes in the European research system. It was the transformational scenarios 3 and 4 that appear to have stimulated the generation of ideas in the focus groups most effectively. Scenario 4, which is a highly transformational and creative scenario, was the most stimulating on four occasions, while Scenario 1 -considered in the ERA scenarios report to be the most familiar and 'baseline' scenario - was the least stimulating scenario in four of the focus groups. So, in the VERA project, those scenarios that represented higher levels of transformation had more capacity to inspire the creation of recommendations, at least in terms of fluency. The analysis supports the idea that the level of transformation of future scenarios with respect to the present, i.e., the level of differentiation of scenarios vis-à-vis "today's context", can influence the

Table 5. VERA Scenarios Stimulation Capacity

Scenario stimulation capacity rate = insights per participant Scenario

1 2 3 4

Society 1.3 (--) 2.8 2.3 3.3 (+)

Academia 0.8 0.6 (--) 1.0 2.3 (+)

Industry 2.3 (--) 2.7 3.3 (+) 2.4

Funders 2.8 (+) 1.82 1.75 1.33 (--)

ERA experts 2.25 (--) 2.7 2.33 3.8 (+)

International 1.3 (--) 2.0 2.0 2.7 (+)

Policymakers 2.7 2.3 (--) 4.5 (+) 3.0

Insights generated in each scenario 75 89 73 92

Total across seven groups 1.97 2.02 2.43 2.70

Source: compiled by the authors.

Figure 1. VERA Scenario Stimulation Rate

3.0

iüi

Scenario 1 Scenario 2 Scenario 3 Scenario 4

Scenario 2

Scenario 3

Scenario 4

Source: compiled by the authors.

number of insights generated by the people stimulated with these scenarios.

Originality Analysis

Expert advice - including that from Foresight activities - often reproduces or only marginally extends conventional or commonly debated policy ideas. Creativity is therefore an important element to promote in Foresight projects, not least because systemic issues typically addressed in Foresight require open thinking, holistic approaches, and novel perspectives. As previously stated, originality is the capacity of proposing more innovative and different ideas than other individuals. It could be argued that personal skills have more influence on the generation of original ideas than the characteristics of the scenarios. The VERA foresight project allows us to explore multiple debates, which should allow us to examine influences that go beyond those associated with the presence of a few highly creative people.

While creativity is often measured on the basis of subjective judgements, we can use a more quantitative approach. This indicator consists of observing, unambiguously, how many "unique" ideas are introduced in each focus group. "Unique" ideas are those which are only introduced by a single type of actor, while not being proposed in another scenario or by a different stakeholder. A comprehensive analysis of data elicited from the focus groups produced a set of these original ('unique' and non-repeated) insights, as reported in Table 7.

Table 6. Scenario Stimulation Capacity

Number of times Ranking points

Scenario most stimulating least stimulating Total

1 (realistic) 1 4 12 28

2 (realistic) 0 2 16

3 (transformational) 2 0 20 42

4 (transformational) 4 1 22

Source: compiled by the authors.

The table indicates that Scenarios 3 and 4 stimulated the originality of insights (original ideas per participant) to a greater extent than Scenarios 1 and 2. Scenario 2 seems to be least effective and scenario 4 was the most liable to promote the most unique insights per participant.

If we analyze scenarios in pairs, scenario 3 and 4 inspired the highest originality in five of the stakeholder groups, whereas scenario 1 and 2 were most inspiring on two occasions. As in the previously described analysis of scenarios' stimulation capacity, the results suggest that transformational scenarios are useful to favor and stimulate the elicitation of more original ideas. There are two distinct, but related, interpretations of these results and they may well both apply. First is that when participants are immersed in more radical, unknown, and unfamiliar contexts, they need to look more actively for non-conventional solutions, due to facing new and different problems. Second is that being presented with different and original elements as compared to today's reality, the participants are themselves inspired to think more creatively. In either case, the development of creative thinking in Foresight processes could be stimulated by confrontation with scenarios that differ substantially from those associated with familiar trends. It remains to be seen whether there is an "optimal" level of transformation - so that too much major transformation, too much novelty, is counterproductive.

In summary, the results suggest that repositioning advisors in highly innovative or disruptive scenarios more effectively stimulates the fluency of ideas and the capacity to propose creative solutions, than does repositioning them in conservative or incremental scenarios. The recommendation phase of Foresight processes is comprehensively explained by the "3R" methodological frame ("Reposition", "Representation", and "Resolution") that is utilized to generate sound advice with future scenarios [Velasco, 2017] (Table 8) Arguably, such an effect on fluency and originality is given by the 'surprise' or 'shock' effect that radical scenarios may promote in the advisors' mindset, in contrast to the predictable reactions associated with 'business-as-usual' situations. But this does not mean that Foresight workshops should avoid confronting participants with conventional contexts. Though the discussion of less surprising circumstances may well elicit more standard recommendations, reflection on these may be valuable for assessing the implications of programs and endorsing plans that are already more likely to be considered or implemented.

Discussion

Recommended Situations for Applying the Method

This study has focused on the ways in which considering different scenarios can influence the recommendations developed from a Foresight exercise. Many studies simply aggregate and list recommendations derived

Table 7. Originality and Stimulation Analysis

Stakeholder Scenario

groups 1 2 3 4

Society 0.00 (--) 1.00 1.33 2.50 (++)

Academia 0.25 (--) 0.38 0.25 (--) 1.50 (++)

Industry 0.75 (--) 1.67 (++) 1.33 1.50

Funders 2.00 (++) 0.55 0.25 (--) 0.33

ERA experts 1.00 (--) 1.56 1.33 2.50 (++)

International 0.67 (--) 1.33 1.67 2.00 (++)

Policymakers 1.67 1.25 (--) 2.75 (++) 1.50

Total across seven groups 1.03 0.98 1.27 1.74

Note: Number of ideas (per participant) generated by a stakeholder group (in the scenario) that have not been mentioned in any other scenario or by another stakeholder.

| Source: compiled by the authors.

from considering alternative scenarios, without indicating how frequently a given recommendation is derived across scenarios, which scenarios do and do not give rise to it. This intelligence could be therefore valuable for addressing complex and systemic problems (the method builds robust recommendations emerging simultaneously across multiple scenarios, thus being applicable to a range of future circumstances) as well as being useful in the event of multiple stakeholders having taken part of strategic debates around those scenarios. The method is also particularly appropriate in circumstances where sponsors and users of the work are liable to require more explication of the ways in which recommendations have been formulated\, and where an effort has been made to develop very different scenarios, especially if some of these are transformative ones. In this sense, the study showed that introducing transformative scenarios in the workshops is helpful in situations where a high number of original insights are required to build sound advice upon them.

Bias

The scenario approach does not presume, in general, that one or another scenario is in some way "correct"

in the sense of accurately forecasting the future - indeed, no scenario can be accurate in this sense. What practitioners usually do is to capture a range of possible futures that is relevant to users and can inform meaningful action to steer developments in positive directions. To reduce bias, our method deploys a variety of scenarios and uses them to reposition participants in a particular set of perspectives. By encouraging the development of those multiple scenarios and providing a structured selection of stakeholders to think strategically on them in focus groups, the method effectively reduces the sort of bias that results from assuming that "business as usual" can effectively encompass the future or that there is only one plausible trajectory of development.

Implications for Theory

The study largely draws on theories and was more concerned with applying them than testing them. In particular, it demonstrates the utility of Guilford's approach [Guilford, 1950, 1967] to measure the outcomes of divergent thinking processes and suggests that a fertile line of research in the policy sciences, planning and knowledge management fields would be to examine how far (sets of) recommendations characterized in terms of the different features highlighted by this approach are readily assimilable by decision-makers, whether they need specific messaging and packaging approaches to be effectively deployed. Exploring the effects of repositioning people in transformative scenarios and assessing the capacity of these scenarios to stimulate the generation of ideas may become an incipient and modest contribution to the theory of creativity that deserves further attention. More empirical studies are yet needed to understand the influence of future scenarios in other manifestations of creative individuals such as the (Guilford's) flexibility of ideas and their level of elaboration.

Implications for Practice

The study implicitly endorses the use of multiple scenarios in Foresight research. However, it should be noted that some scenario studies focus on aspirational

Table 8. "3R" Methodological Frame for Sound Advice

Factor Description Effect on Foresight sound advice

Reposition This factor refers to the process whereby participants situate their mindsets in a hypothetical future context and adopt decisions or devise strategies as if they were living or immersed in these contextual circumstances. Repositioning participants in highly transformed scenarios stimulates their creativity in particular by facilitating the generation of more numerous and original ideas. Modulate the number and originality of ideas by repositioning participants in innovative future contexts

Representation This factor relates to the composition of advisory panels and multi-stakeholder workshops in the foresight processes. The presence of different actors and areas of knowledge within these panels has an important influence on the variety and flexibility of themes/perspectives considered by the participants to find solutions in problem-solving situations. Adapt participants' perspectives with an adequate representation of actors

Resolution This is associated to the intervention needed to elaborate upon the advice discourse from the initial insights generated by participants 'repositioned' into incremental or transformational scenarios. Such interventions are supported by argumentation, which influences the type of advice generated and the level of elaboration of the final recommendations. Increase the quality and soundness of advice with argumentation rules

Source: compiled by the authors basing on [Velasco, 2017].

scenarios (these may or may not be thoroughly transformational), while other methods, such as roadmap-ping, typically deploy just a single scenario. On the practical utilization of scenarios we would suggest that when resources permit, (a) these approaches will be enriched by the preceding exploration and repositioning in multiple scenarios (sometimes an aspirational scenario may be composed from a selection of these) and (b) the full range of recommendations stemming from multiple scenario analysis will enrich those being thrown up in the course of roadmapping and similar efforts. To work on these scenarios, we encourage Foresight practitioners to get a balanced representation of stakeholders, whose coincident insights across focus groups and plausible futures will facilitate the elaboration of more robust advice. As with all scenario work, face-to-face workshops such as those described in our case study require time and careful preparation, ideally with detailed "scripts" for facilitators. In particular, our method implies that the details of scenarios are clearly conveyed, as they describe contexts in which participants are asked to be repositioned. In this respect, the capacity for adaptation, flexibility, and critical thinking are requested for participants to get the most out of repositioning them in those future contexts.

While it is undoubtedly important for participants to feel comfortable with the scenarios they are using, especially where it comes to working with a vision of the future they can believe as being plausible, there is however a danger of proposing not sufficiently transformed scenarios, so that they find them close to the "most likely" situations, and thus losing the effect on boosting creativity that the method pursues.

Conclusions: Lessons from the Case Study and Implications for Further Research and Practice

Foresight is used to support decision-making around important and complex issues. Although sometimes just providing background intelligence, foresight activities are also frequently asked to provide concrete sets of recommendations. Beyond serving as good presentational devices to show that many aspects of the future are open, future scenarios are used in Foresight activities to support the development of recommendations that can help decision-makers to initiate actions that affect an extended present.

The examination of the VERA project presented here has tackled a gap in the literature: the absence of empirical analysis of how scenario methodologies can shape the advice generated by Foresight activities. It throws some light on the factors that can promote creative knowledge in Foresight workshops.10 Specifically,

it demonstrates that a method for repositioning people in transformational future scenarios can contribute to the fluency and creativity of ideas - at least, the repositioning did so in this one case study. Since the VERA project systematically replicated a substantial number of workshops, we have grounds to think that these results are likely to be significant for other exploratory scenario-based projects. Indeed, similar results could be probably found using other foresight tools, for example wild card generation activities that do not involve a full scenario analysis. Hopefully, the present study provides stimulus for further research seeking to accumulate evidence on these issues. To be sure, many Foresight activities are conducted under such time pressures, and with such budget limitations, practitioners find it hard to mount a systematic exploration of such themes. But it should be possible to collect indicative data from many scenario workshop studies (this may often be less systematic, but could still be indicative). Furthermore, those involved in Foresight education and training could mount experimental studies using their student or participant groups. Many topics for further research are thrown up by the present study (and some further analyses of the VERA results are to be the subject of forthcoming papers by the authors). Here are some examples:

• The case study involved one type of scenario methodology, for instance - how similar would the results be for a study applying different scenario tools?11

• What if the actors involved in the recommending phase are also those involved in the initial scenario development as is often the case - does the process of creatively constructing scenarios affect the depth of repositioning and the extent to which this leads to creative insights?

• Are the results affected by the topic of the Foresight study - would an examination of particular technologies or social issues engender similar patterns of fluency and originality?

• How do differences in individual creativity and in the prior knowledge/expectation of different stakeholder groups affect outcomes? The results presented in the tables above suggest that (a) the influence of scenarios of different types varies across such groups, who (b) themselves appear to vary in terms of fluency and originality. This paper has been more concerned with the overall pattern of influence of the scenario types; but if there are consistent differences in such influence across different stakeholder groups, this may also need to be understood and taken into account in Foresight design.

This study also has implications for Foresight practitioners. In the Foresight design process, they decide not

10 Some of these factors, such as the influence of facilitators or the imagination of participants, were partially considered by [Dufva, Ahlqvist, 2015].

11 For example, conclusions from [Miles et al., 2016] contrast this 2*2 approach with "archetypes" and "aspirational" approaches.

just on scenario methodology, but also on the extent of elaboration and 'ornamentation' of scenarios. Although exhaustive scenario descriptions can provide varied themes for debate, they could unintentionally also lead to participants' discussions deviating (for better or worse) from sponsors' specific areas of interest. In fact, expectations about sponsors are liable to play a major role in the formulation of advice based on the recommending phase. Practitioners might consider what expectations to build into the process. For example, should it be stressed that sponsors may not seek "too much" originality, that they may prefer to play it safe and avoid thinking about transformational scenarios? But such "safety" is highly risky when the long term is involved. Practitioners might instead try to focus more on ways of demonstrating that unpalatable plausible futures do require consideration. If such visions are to be taken seriously, the insights from repositioning may be valuable tools for illustrating things that cannot be ignored.

Again, sponsors may also resist large numbers of recommendations and the Foresight team may need to find ways of prioritizing recommendations that avoid relegating too many ideas to categories of minor interest. One approach that is often helpful is to consider bundling up individual actions into "joined up" programs of activity. This is in any case helpful for building on synergies and avoiding inconsistencies and even contradictions across policy efforts, as highlighted in the literature on "policy mixes" [Flanagan et al., 2011]. Practitioners do not always have to undertake such synthesis as a heroic effort after the workshops have concluded but can - if time permits - engage the participants in relevant discussions. We speculated above that the effect of repositioning might not be linear across different degrees of scenario transformation and this is another topic for research as well as the anecdotal evidence of practitioners. There could be dampening effects of repositioning people in too radical transformations. For example, total disaster scenarios, or ones that are dependent on extreme wild cards, might lead participants to think not much can be done. This could be a matter of believing that the scenario would mean a paralysis of decision-making -or simply that here and now, decision-makers would be unwilling to listen to any recommendations arising

from such a future vision. It remains therefore to be studied whether there is an "optimal" level of scenario transformation and demonstrate that, perhaps, too much major transformation and novelty in scenarios may be counterproductive and bring about a negative effect on the generation of ideas. Another design-related topic for further study involves the assistance that facilitators could give to help participants to easily reposition their mindsets in the proposed future contexts. Facilitators are liable to vary in ability and such skills as imagination, motivation, agility of thinking, and empathy. But some of these capabilities can be gained or enhanced through appropriate training and guidance as to ways of prompting discussion, defusing arguments, and the like can be generated and applied. It would be useful to have more communication across Foresight practitioners as to ways of promoting and gaining value from the repositioning process. One aspect of repositioning that could certainly benefit from more research is the use of role-playing (the "persona" approach, for example, and simulation gaming more generally) as a way of increasing the immersion of participants in specific scenarios [Fergnani, 2019]. It is at least worth researching the idea that debates about actions from participants who are playing different roles, could deepen the analysis of scenarios and lead to even more creative insights (including those involving competition and contest, cooperation, and coevolution).

This paper is intended to contribute to the accumulation of knowledge as to the methodological basis of Foresight activities and to open the black box related to the recommendation phase of such activities. We hope to open a discussion on why and how future repositioning can be incorporated as a variable in the design of foresight activities and future-oriented critical issues analysis.

We would like to thank the European Commission, which funded the VERA project (European Union's FP7 programme for research, technological development and demonstration, under grant agreement no 290705), and the Fraunhofer Institute for Systems and Innovation Research ISI (Dr. Stephanie Daimer) for the project coordination. Miles' contribution to this paper was prepared within the framework of the Basic Research Program of the National Research University Higher School of Economics."

References

Alvarez A. Ritchey T. (2015) Applications of General Morphological Analysis: From Engineering Design to Policy Analysis. Acta Morphologica

Generalis, 4(1), 1-40. http://www.amg.swemorph.com/pdf/amg-4-1-2015.pdf, accessed 19.08.2020. Amabile T.M. (1983) The social psychology of creativity, New York: Springer.

Budescu D.V., Rantilla A.K., Yu H., Karelitz T.K. (2003) The effects of asymmetry among advisors on the aggregation of their opinions.

Organisational Behavior and Human Decision Processes, 90(1), 178-194. Bussey M. (2014) Concepts and effects: Ordering and practice in foresight. Foresight, 16(1), 1-16. https://doi.org/10.1108/FS-04-2013-0017 Caplan S.E., Samter W. (1999) The role of facework in younger and older adults' evaluations of social support messages. Communication

Quarterly, 47, 245-264. https://doi.org/10.1080/01463379909385558 Carleton T., Cockayne W, Tahvanainan A.-J. (2015) Playbook for Strategic Foresight and Innovation, Stanford, CA: DesignX Lab - Foresight, Center for Design Research, University of Stanford.

Cross R., Borgatti S.P., Parker A. (2001) Beyond answers: Dimensions of the advice network. Social Networks, 23(3), 215-235. https://doi.

org/10.1016/S0378-8733(01)00041-7 Da Costa O., Warnke P., Scapolo F., Cagnin C. (2008) The impact of foresight on policy making: Insights from the FOR- LEARN mutual

learning process. Technology Analysis and Strategic Management, 20(3), 369-387. https://doi.org/10.1080/09537320802000146 Daimer S., Gheorghiu R., Ordonez-Matamoros H.G., Laredo P., Giesecke S., Popper R. Loikkanen T., Molas-Gallart J., Haegeman K., Degelsegger A. (2015) VERA Final Report. Report of the Forward Visions on the European Research Area (VERA) project, funded by the European Union's FP7 programme for research, technological development and demonstration, Brussels: European Commission. https://ris.utwente. nl/ws/files/5135969/VERA_Final_Report_2015-06-30_public.pdf, accessed 28.08.2020. Dalal R.S., Bonaccio S. (2010) What types of advice do decision-makers prefer? Organisational Behavior and Human Decision Processes, vol. 112,

11-23. https://doi.org/10.1016Zj.obhdp.2009.11.007 Dator J. (2009) Alternative Futures at the Manoa School. Journal of Futures Studies, 14(2), 1-18. Dator J., Rodgers S. (1991) Alternative futures for the state courts of2020, Chicago, IL: American Judicature Society. De Jouvenel B. (1967) The Art of Conjecture, London: Weidenfeld and Nicolson.

De Smedt P. (2013) Interactions between foresight and decision making. In: Participation and Interaction in Foresight: Dialogue, Dissemination and Visions (eds. K. Borch, S. Dingli, M.S. J0rgensen), Cheltenham: Edward Elgar, pp. 17-34. https://doi.org/10.4337/9781781956137.00008 UK Government (2009) Multi-criteria analysis: A manual, Wetherby: Department for Communities and Local Government. https://assets. publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/191506/Mult-crisis_analysis_a_manual.pdf, accessed 18.08.2020.

Dorfler V, Stierand M. (2020) (eds.) Handbook of Research Methods on Creativity, Cheltenham: Edward Elgar Publishing.

Dufva M., Ahlqvist T. (2015) Knowledge creation dynamics in foresight: A knowledge typology and exploratory method to analyse foresight

workshops. Technological Forecasting and Social Change 94, 251-268. https://doi.org/10.1016/j.techfore.2014.10.007 Eriksson A., Weber M. (2008) Adaptive foresight: navigating the complex landscape of policy strategies. Technological Forecasting and Social

Change, 75(4), 462-482. https://doi.org/10.1016/j.techfore.2008.02.006 European Commission (2002) Thinking, debating and shaping the future: Foresight for Europe . Final report from a High Level Expert Group for the European Commission. Brussels: European Commission Directorate-General for Research Unit RTD-K.2. https://www.researchgate.net/ publication/262013061_Thinking_debating_and_shaping_the_future_Foresight_for_Europe, accessed 28.08.2020. Fergnani A. (2019) The future persona: A futures method to let your scenarios come to life. Foresight, 21(4), 445-466. https://doi.org/10.1108/ FS-10-2018-0086

Fisher J.D., Nadler A., Whitcher-Alagna S. (1982) Recipient reactions to aid. Psychological Bulletin, 91 (1), 27-54. https://psycnet.apa.org/ doi/10.1037/0033-2909.91.1.27

Flanagan K., Uyarra E., Laranja M. (2011) Reconceptualising the 'policy mix' for innovation. Research Policy, 40(5), 702-713. https://doi. org/10.1016/j.respol.2011.02.005

Gabina J. (1995) El futuro revisitado: La reflexion prospectiva como arma de estrategia y decision, Barcelona, Marcombo - Boixareu Editores. Georghiou L., Cassingena Harper J., Keenan M., Miles I., Popper R. (eds.) (2008) The Handbook of Technology Foresight. Concepts and Practice

(PRIME Series on Research and Innovation Policy), Cheltenham: Edward Edgar Publishing Limited. Gibbons A.M. (2003) Alternative forms of advice in natural decision settings. Unpublished master's thesis, University of Illinois, Urbana-Champaign.

Glover J.A., Ronning R.R., Reynolds C.R. (1989) (ed.) Handbook of Creativity. Boston, MA: Springer. Godet M. (1992) De l'Anticipation a ' l'Action: Manuel de Prospective et de Stratégie, Paris: Dunod.

Goldsmith D.J. (1994) The role of facework in supportive communication. In: Communication of social support: Messages, interactions,

relationships, and community (eds. B.R. Burleson, T.L. Albrecht, I.G. Sarason), pp. 29-49. Thousand Oaks, CA: Sage Publications. Guilford J.P. (1950) Creativity. American Psychologist, 5(9), 444-454. https://psycnet.apa.org/doi/10.1037/h0063487 Guilford J.P. (1967) The nature of human intelligence, New York: McGraw-Hill.

Haegeman K., Pérez S., Marinelli E., Warnke P., Teufel B., Degelsegger A., Weiss G. (2012) Communication Strategy, JRC/IPTS-Fraunhofer ISI-

ZSI Report of the Forward Visions on the European Research Area (VERA) project. Brussels: European Commission. Harper J., Pace G. (2007) Creative processes in policy making: A case for context in foresight. In: Creative Thinking: Designing Future Possibilities (ed. M.S. Dingli.), pp. 1-11, Valetta: Malta University Press. http://www.cs.um.edu.mt/gordon.pace/Teaching/Foresight/Papers/context.pdf, accessed 19.04.2021.

Havas A. (2005) Terminology and Methodology for Benchmarking Foresight Programmes (ForSociety Project, Tasks 1.4 and 3.1), Brussels:

European Commission. https://mpra.ub.uni-muenchen.de/77791/, accessed 28.08.2020. Heath C., Gonzalez R. (1995) Interaction with others increases decision confidence but not decision quality: Evidence against information collection views of interactive decision-making. Organisational Behavior and Human Decision Processes, 61, 305-326. https://doi.org/10.1006/ obhd.1995.1024

Huang W-C., Lee Y.-Y. (2016) Strategic Planning for Land Use under Extreme Climate Changes: A Case Study in Taiwan. Sustainability, 8(1),

53. https://doi.org/10.3390/su8010053, accessed 19.08.2020. Inayatullah S. (2004) The causal layered analysis (CLA) reader: Theory and case studies of an integrative and transformative methodology. Taipei, Taiwan: Tamkang University Press.

Johnson T.R., Budescu D.V., Wallsten T.S. (2001) Averaging probability judgments: Monte Carlo analyses of diagnostic value. Journal of

Behavioral Decision Making, 14, 123-140. https://doi.org/10.1002/bdm.369 Kahane A. (2012) Transformative Scenario Planning: Working Together to Change the Future, San Francisco, CA: Berrett-Koehler. Keenan M., Popper R. (2008) Comparing foresight 'style' in six world regions. Foresight, 10(6), 16-38. https://doi.org/10.1108/14636680810918568 Kincaid M., Duffus L. (2004) Learning, Thinking and Creativity, New York: IDES.

Kuwahara T. (1996) Technology foresight in Japan: A new approach in methodology and analysis. STI Review, 11, 51-70. Ladkin D. (2004) Action research. In: Qualitative Research Practice (2004) (eds. C. Seale, G. Gobo, J.F. Gubrium, D. Silverman.), London: Sage Publications, pp. 478-490.

Malaska P. (2001) A Futures Research Outline of a Post-Modern Idea of Progress. Futures, 33, 225-243. https://doi.org/10.1016/S0016-3287(00)00069-0

Mardani A., Jusoh A., Zavadskas E.K., Cavallaro F., Khalifah Z. (2015) Sustainable and Renewable Energy: An Overview of the Application of Multiple Criteria Decision Making Techniques and Approaches. Sustainability, 7(10), 13947-13984. https://doi.org/10.3390/su71013947 Martin L., Wilson N. (2018) (eds.) The Palgrave Handbook of Creativity at Work, London: Palgrave Macmillan.

Masini E. (1982) Reconceptualizing the Future: A Need and a Hope. World Futures Society Bulletin, 16(6), 1-8. http://master-foresight-innovation.fr/wp-content/uploads/2012/06/EMasiniReconceptualizingFutures.pdf, accessed 19.04.2021.

McKernan J. (1996) Curriculum Action Research: A Handbook of Methods and Resources for the Reflective Practitioner (2nd ed.), London: Kogan Page.

Medina J. (1999) The research on future images and visions: Need for a strategic alliance between the futures studies and social sciences. International Review of Sociology, 9(3), 333-348. https://doi.org/10.1080/03906701.1999.9971321 Miles I. (2008) From Futures to Foresight. In: The Handbook of Technology Foresight. Concepts and Practice (PRIME Series on Research and Innovation Policy) (eds. L. Georghiou, J. Cassingena Harper, M. Keenan, I. Miles, R. Popper), Cheltenham: Edward Edgar Publishing Limited, pp. 24-40.

Miles, I. Cassingena Harper, J., Georghiou, L., Keenan, M., and Popper, R. (2008a) "The Many Faces of Foresight", In: The Handbook of Technology Foresight. Concepts and Practice (PRIME Series on Research and Innovation Policy) (eds. L. Georghiou, J. Cassingena Harper, M. Keenan, I. Miles, R. Popper), Cheltenham: Edward Edgar Publishing Limited, pp. 3-23. Miles I., Cassingena Harper J., Georghiou L., Keenan M., Popper R. (2008b) New Frontiers: Emerging Foresight. In: The Handbook of Technology Foresight. Concepts and Practice (PRIME Series on Research and Innovation Policy) (eds. L. Georghiou, J. Cassingena Harper, M. Keenan, I. Miles, R. Popper), Cheltenham: Edward Edgar Publishing Limited, pp. 400-417. Miles I., Saritas O., Sokolov A. (eds.) (2016) Foresight for Science, Technology and Innovation, Berlin: Springer.

Mitchell R.K., Agle B.R., Wood D.J. (1997) Toward a theory of stakeholder identification and salience: Defining the principle of who and what

really counts. Academy of Management Review, 22, 853-886. https://doi.org/10.5465/amr.1997.9711022105 Paulus P.B. (2000) Groups, Teams, and Creativity: The Creative Potential of Idea-generating Groups. Applied Psychology: An International

Review, 49(29), 237-262. https://psycnet.apa.org/doi/10.1111/1464-0597.00013 Paulus P.B., Nijstad B.A. (2019) (eds.) The Oxford Handbook of Group Creativity and Innovation, Oxford: Oxford University Press. Popper R., Keenan M., Miles I., Butter M., Saintz G. (2007) Global foresight outlook. Mapping foresight in Europe and the rest of the world, Brussels: European Commission.

Popper R. (2008a) How are foresight methods selected? Foresight, 10(6), 62-89. https://doi.org/10.1108/14636680810918586 Popper R. (2008b) Foresight Methodology. In: The Handbook of Technology Foresight. Concepts and Practice (PRIME Series on Research and Innovation Policy) (eds. L. Georghiou, J. Cassingena Harper, M. Keenan, I. Miles, R. Popper), Cheltenham: Edward Edgar Publishing Limited, pp. 44-90.

Popper R. (2009) Mapping Foresight: Revealing how Europe and other world regions navigate into the future. Luxembourg: Publications Office of

the European Union, European Commission. Popper R., Velasco G., Edler J., Amanatidou E., Miles I. (2015a) ERA Open Advice: Forward Visions on the European Research Area (VERA project report, funded by the European Union's FP7 programme for research, technological development and demonstration), Manchester: The University of Manchester. http://eravisions.archiv.zsi.at/page/22/attach/ERA_Open_Advice.pdf, accessed 28.08.2020. Popper R., Edler J., Velasco G., Amanatidou E., Miles I. (2015b) ERA Strategy Map (Report for the EC funded project VERA on Visions for the European Research Area, funded by the European Union's FP7 programme for research, technological development and demonstration), Manchester: The University of Manchester. http://eravisions.archiv.zsi.at/page/22/attach/VERA_WP5_D5_2_ERA_Strategy_ Map_26_6_2015.pdf, accessed 28.08.2020. Popper R., Popper M., Velasco G. (2020) Sustainable Innovation Assessment and Management Framework: Principles, Methodology and Practice. In: Governance and Management of Sustainable Innovation: Learning from Experience to Shape the Future (eds. M. Martini, R. Holsgens, R. Popper), Cham: Springer, pp. 3-39. https://doi.org/10.1007/978-3-030-46750-0_1 Ramos J. (2017) Linking Foresight and Action: Toward a Futures Action Research. In: The Palgrave International Handbook of Action Research (eds. L.L. Rowell, C.D. Bruce, J.M. Shosh, M.M. Riel), London: Palgrave-Macmillan, pp. 823-842. https://doi.org/10.1057/978-1-137-40523-4_48

Rietzschel E., Nijstad B., Stroebe W. (2010) The selection of creative ideas after individual idea generation: Choosing between creativity and

impact. British Journal of Psychology, 101, 47-68. https://doi.org/10.1348/000712609X414204 Ringland G. (2006) Scenario Planning. Hoboken, N.J.: Wiley.

Salo A., Cuhls K. (2003) Technology Foresight - Past and Future. Journal of Forecasting, 22(2-3), 79-82. https://doi.org/10.1002/for.846 Scapolo F., Miles I. (2006) Eliciting Experts' Knowledge: A Comparison of Two Methods. Technological Forecasting and Social Change, 73(6),

679-704. https://doi.org/10.1016Zj.techfore.2006.03.001 Schrah G.E., Dalal R.S., Sniezek J.A. (2006) No decision-maker is an island: Integrating expert advice with information search. Journal of

Behavioral Decision-Making, 19, 43-60. https://doi.org/10.1002/bdm.514 Slaughter R. (1995) The foresight principle, Westport, CT: Adamantine Press.

Soll J. B. (1999) Intuitive theories of information: Beliefs about the value of redundancy. Cognitive Psychology, 38, 317-346. https://doi. org/10.1006/cogp.1998.0699

Staton M. (2008) Monstrous foresight. In: (Eds. C. Cagnin, M. Keenan, R. Johnston, F. Scapolo, R. Barré), Future-oriented Technology Analysis.

Strategic Intelligence for an Innovative Economy, Berlin, Heidelberg: Springer, pp. 53-68. https://doi.org/10.1007/978-3-540-68811-2_5 Sternberg R.J. (1985) Beyond IQ: A Triarchic Theory of Intelligence, Cambridge: Cambridge University Press. Sternberg R. (1998) (ed.) Handbook of Creativity, Cambridge: Cambridge University Press.

Teufel B., Erdmann L., Schirrmeister E., Daimer S., Laredo P., Schoen A., Robinson D.K.R., Loikkanen T. (2013) ERA Scenario Report (Report of the Forward Visions on the European Research Area (VERA) project). Brussels: European Commission. http://eravisions.archiv.zsi.at/ page/22/attach/WP3_ERA_Scenario_report_final_28052015.pdf, accessed 28.08.2020. Torrance E.P. (1968) Education and The Creative Potential, Minneapolis: University of Minnesota Press.

Torrance E.P. (1974) Torrance Tests of Creative Thinking: Directions Manual and Scoring Guide (Verbal Test Booklet A), Lexington, MA: Personnel Press.

Uotila T., Melkas H., Harmaakorpi V. (2005) Incorporating futures research into regional knowledge creation and management. Futures, 37,

849-866. https://doi.org/10.1016/jfutures.2005.01.001 Velasco G. (2017) Understanding the generation of research and innovation policy advice with foresight processes (PhD Thesis), Manchester: The

University of Manchester. https://www.research.manchester.ac.uk/portal/files/57431200/FULL_TEXT.PDF, accessed 28.08.2020. UK Government (2017) Futures Toolkit: Tools for strategic futures for policy-makers and analysts. London: Cabinet Office and Government Office

for Science. https://www.gov.uk/government/publications/futures-toolkit-for-policy-makers-and-analysts, accessed 18.08.2020. Weisburg R.W. (1986) Creativity: Genius and Other Myths, New York: Freeman.

Yaniv I. (2004) Receiving other people's advice: Influence and benefit. Organisational Behavior and Human Decision Processes, 93, 1-13. https://

doi.org/10.1016/j.obhdp.2003.08.002 Yin R.K. (2014) Case Study Research: Design and Methods (5th ed.), London: SAGE.