CC BY-NC-ND
115
Agile Roadmapping: An Adaptive Approach to Technology Foresight
Eoin O'Sullivan
Director, Centre for Science, Technology & Innovation Policy, [email protected]
Rob Phaal
Director of Research, Centre for Technology Management, [email protected] University of Cambridge, 17 Charles Babbage Rd, Cambridge CB3 0FS, UK
Charles Featherston
Head of Futures, [email protected]
UK Government Office of Science, 10 Victoria Street, London, SW1H 0NN, UK
Abstract
Technology roadmapping has become an important foresight tool for science, technology, and innovation (STI) policy and technology strategy development. There are, however, challenges in translating evidence from foresight into the strategies of STI agencies and the planning of research & technology development (RTD) organizations. While the foresight evaluation literature identifies methodological issues related to evidence granularity, scope, and stakeholder confidence, there is limited guidance on how to ensure roadmapping outputs are strategically relevant, appropriately detailed, and credible. This paper highlights the potential of using structured visual roadmapping frameworks to anticipate potential strategic foresight evidence failures and using the adaptive and iterative nature of roadmapping processes to address them. In this paper, we distinguish between: the roadmapping framework 'canvas'; the foresight evidence captured on the canvas; the process of generating the evidence; and any final
strategic plan developed using that evidence (with goals, milestones, actions, etc). We investigate efforts to use the roadmapping canvas as a research tool and diagnostic to explore emerging technology trajectories and innovation 'pathways'. We demonstrate that key patterns of evidence distribution on the roadmapping canvas have the potential to reveal where further evidence may need to be gathered, or where further triangulation of stakeholder perspectives may be required. We argue that by adaptively addressing these patterns at key stages within the roadmapping process (and appropriately re-scoping, re-prioritizing, and re-focusing foresight effort and resources), the granularity, coverage, and consensus of the roadmapping evidence can be greatly enhanced. We conclude the paper by summarizing a set of novel principles for adaptive agile roadmapping, reflecting on the implications for foresight more generally, and outlining a future research agenda to test and refine this approach to agile foresight.
Keywords: technology roadmapping, foresight methodology, emerging technology strategy, STI policy
Citation: O'Sullivan E., Phaal R., Featherston Ch. (2021) Agile Roadmapping: An Adaptive Approach to Technology Foresight. Foresight and STI Governance, 15(2), 65-81. DOI: 10.17323/25002597.2021.2.65.81
© 2021 by the authors. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Technology roadmapping has become an important foresight tool for science, technology, and innovation (STI) policy and government R&D strategy. There are, however, challenges in translating evidence from foresight into the strategies of STI agencies and research & technology development (RTD) organizations. In this paper, we explore the opportunities to take advantage of the distinctive properties of technology roadmapping (frameworks and processes) to navigate the complexity of innovation system- or industry-level foresight and to enhance the relevance, granularity, and credibility of strategic foresight evidence. Although the origins of roadmapping lie in firm-level strategy development [Kerr, Phaal, 2020], technology roadmapping has become an important foresight tool for STI policy [Cho et al., 2016], where it is used, for example, to analyze technology innovation dynamics as part of public-private industrial sector-level strategic planning [Baldi, 1996; Harrell et al., 1996; Nimmo, 2013] or national technology foresight [de Almeida et al., 2015; Hussain et al., 2017; Saritas, Oner, 2004]. There are, however, variations in emphasis (and challenges) when applying the approach at the innovation system or industrial levels. In contrast to firm-level technology roadmapping, innovation system-level exercises may involve even more complex analyses (with a more diverse set of stakeholders), more complex and longer term innovation system dynamics, and broader socio-political trends and drivers [Cho et al., 2016; Isenmann, 2008; Schuh et al., 2013]. Furthermore, unlike firm-level road-maps, where the stakeholders commissioning the roadmap are often the same ones that will use the outputs, government-commissioned foresight exercises are often intended to generate an evidence base to be used by a range of different public sector actors (e.g., research and innovation agencies, research and technology organizations) for their individual strategic purposes [Cho et al., 2016; Schuh et al., 2013]. In this context, the general challenge of ensuring outputs have the right level of detail, scope, and stakeholder confidence [Lee et al., 2012; Schuh et al., 2013] to support the strategy development needs of the foresight evidence users can become more difficult. Despite the increasing popularity of innovation system-level foresight, however, there is limited guidance for ensuring the effectiveness and impact of STI roadmapping exercises [Kostoff, Schaller, 2001; Oliveira, Fleury, 2015]. Similarly, there is little guidance on characterizing or reporting on the limitations of a roadmapping evidence. Indeed, STI roadmaps rarely contain statements on any limitations of the underpinning data or analysis, from which the academic or practitioner foresight communities might learn.
This paper sets out to address this issue by reviewing the foresight evaluation literature and broader technology roadmap-ping literature for insights and approaches to identifying and addressing the limitations of foresight evidence and potential sources of error. We then translate these insights into the visual language of roadmapping and explore the relationships with particular patterns of foresight evidence distribution on the roadmapping canvas. In particular, we demonstrate that unique qualities of roadmapping as a foresight tool - the structured visual representation of evidence, the attention to innovation pathways, and the scalable/systemic nature of the framework - mean that key patterns of data distribution on the roadmapping canvas can help anticipate potential sources of foresight evidence failure. Furthermore, we argue that by adaptively addressing these patterns at key stages within the roadmapping process - and, where appropriate, re-scoping, re-prioritizing, and re-focusing the roadmapping effort and
resources - the granularity, coverage, and consensus of the roadmapping evidence should be greatly enhanced.
The Challenges of Translatine(Innovation System-Level) Foresight andTechnology Roadmapping Evidence
In this section, we explore the challenges of technology foresight at the innovation system- or industry-level. In particular, we examine the difficulties of ensuring foresight outputs are accepted and used by STI stakeholders as part of the development of research & innovation strategies. In this context, we review what the foresight and technology roadmapping literature tells us about sources of error, evidence limitations, and barriers to outputs being used. We also review literature related to the evaluation of foresight studies and roadmaps, including any guidance or principles on how to improve road-mapping processes and performance.
In contrast to firm-level technology roadmapping, industry sector-level or innovation system-level studies typically involve more complex analyses, reflecting the system complexity of the innovation dynamics being studied. Consequently, such studies involve greater effort and resources. As Cho et al point out: "With respect to procedures, scope, resources, and time spending, industry roadmap requires much more than the corporate one. While corporate roadmaps target a particular technology and product, industrial roadmaps sometimes deal with wider R&D issues associated with a high level of emerging technology and product trends in the industry" [Cho et al., 2016]
In common with the general characteristics of foresight studies, roadmapping analyses supporting innovation strategies are intrinsically "complex, uncertain and conflicting" in nature [Saritas, Oner, 2004]. Foresight analyses of emerging technologies (and associated challenges and opportunities) are increasingly multidisciplinary, requiring input from a multiplicity of experts and stakeholders, from a range of disciplines and organizations, all with potentially different perspectives. Furthermore, these actors will have a range of different interests, values, incentives and, consequently, different priorities [Saritas, Oner, 2004].
This complexity has further consequences for the effective commissioning and designing of foresight analyses. Policymakers commissioning technology roadmaps (or other foresight analyses) may not anticipate or understand the scale of system complexity, the diversity of stakeholder perspectives, or the interdependence of innovation activities, events and policies [Saritas, Oner, 2004].
There are a range of challenges in developing and applying STI policy evidence generated from foresight [Georghiou, Keenan, 2006; Martin, Irvine, 1990; Saritas, Oner, 2004]. Alongside issues related to organizational absorptive capacity and alignment with policy life cycles, there are also important methodological issues - related to the precision, relevance, and credibility of foresight evidence - which can inhibit its translation into STI strategy development. It is these challenges - and approaches to overcoming them - that are the main focus of this paper.
As pointed out by Georghiou and Keenan, "Foresight is not always tuned to the needs of recipients and hence, to extend the analogy, the signal may be obscured by noise and not picked up. Information needs to be presented in such a way that policy/ strategy mechanisms can receive and absorb it." [Georghiou, Keenan, 2006]. This is an issue facing policy makers, R&D
agency officials, and also public-private research & technology development (RTD) organizations [Salmenkaita, Salo, 2004].
Many foresight exercises often fail to reliably generate evidence in a format that can be effectively used by policymakers or that is sufficiently focused on 'specific [policy] questions' [Day, 2013; Kunseler et al., 2015].
One of the primary reasons for the lack of impact of foresight analysis is "the treatment of foresight and its implementation as separate processes without serious attempts to build bridges between or to link the two" [Georghiou, Keenan, 2006]. The analysis of UK foresight highlights deficiencies in "establishing the link between requirements and eventual implementation and starting initiatives, [as] major problems that the UK foresight studies experienced" [Saritas, Oner, 2004].
Foresight exercises intended to inform strategy development for emerging technology research and innovation investment require evidence on the innovation pathways of emerging technologies (between research and eventual applications). These pathways "are many, not necessarily linear and require an enormous amount of data for any attempt to link research with application. Substantial time and effort are required to portray these links as accurately as possible, and substantial thought is necessary to articulate and portray the massive amount of data in a form comprehensible to potential investors" [Saritas, Oner, 2004].
Given this complexity, it is not possible at the commissioning stage of a foresight exercise to fully anticipate how complex the innovation dynamics of a particular technology might be. One cannot, ab initio, know what level of microtechnical innovation detail may be required in order to identify potential technology 'innovation pathways'. One cannot precisely anticipate the level of consensus or disagreement among innovation stakeholders (regarding key events, trends, barriers, priorities, etc). In terms of foresight process, therefore, it is not possible to fully identify the right cohort of foresight exercise participants (in terms of the sampling of stakeholder perspectives and expertise) or strike the right balance of exercise scope and resources (in order to ensure the outputs are sufficiently granular, focused, and credible to be actionable). This suggests that the strategic foresight analysis of highly complex technology innovation systems must be both adaptive and iterative, to ensure outputs that are relevant, usable, and trusted by 'users'.
This poses important methodological questions about how to both configure technology foresight exercises with strategic evidence requirements of the STI policy users in mind; and how to effectively monitor and regulate the collection of strategic foresight evidence to ensure these requirements are met. In the following section we explore how technology roadmapping analysis and process frameworks offer the potential to:
• facilitate the configuration of foresight exercises to address users' strategic evidence requirements;
• reveal when evidence gathered on particular innovation activities or dynamics may require more granular detail, stakeholder input, or focus on particular innovation system elements or phases;
• offer structured decision points within the foresight exercise for adaptation in response to emerging findings - opportunities for re-scoping, re-prioritizing, and re-focusing foresight effort and resources.
The Technology Roadmapping Canvas: A Diagnostic Tool for Exploring Innovation System Dynamics
In this section, we review the use of technology roadmapping as a foresight and strategy tool - highlighting its distinctive features and functions (in comparison with other foresight tools). We also explore the application of the roadmapping frameworks as a research tool, in particular its use in structuring analyses of emerging technology innovation dynamics and sociotechnical change. In this context, we explore how certain features of the roadmapping canvas may offer the potential for its application as a diagnostic tool to examine the sufficiency, efficacy, and credibility of foresight evidence.
Comparison of Roadmapping and (Other) Foresight Methods
Technology roadmapping is one of a large number of foresight-related methods.1 Following Park et al. we will define roadmapping as:
"A process that mobilizes structured systems thinking, visual methods (e.g. roadmap canvas) and participative approaches to address organizational challenges and opportunities, supporting communication and alignment for strategic planning and innovation management within and between organizations at firm and sector levels" [Park et al., 2020].
As with national level foresight, roadmapping exercises typically convene "people representing different expertise and interests, and use instruments and procedures that allow participants to simultaneously adopt a micro view of their own disciplines and a systems view of overriding or shared objectives" [Coates et al., 2001].
Roadmapping often integrates outputs and insights from other foresight analyses to provide evidence for strategy development and planning "as a tool, defining paths to meet future requirements, roadmaps can assist to connect the future's requirements and today's research areas" [Saritas, Oner, 2004]. As highlighted by Popper, "the bridge between foresight and planning is sometimes achieved with methods like roadmapping" [Popper, 2008].
Because of its role in supporting strategy development, roadmapping is often considered a 'downstream' foresight tool (by contrast with more exploratory methods such as horizon scanning). Although the final output of roadmapping analyses may identify key planning milestones and options for strategic goals, the roadmapping process typically contains an opportunity-scanning phase. Roadmapping can, therefore, be both exploratory and normative, capturing both types of evidence within a single integrating system framework) [Barker, Smith, 1995; Cho et al., 2016; Kappel, 2001]. More generally, roadmapping analyses can address a range of key foresight success factors, for example: "Be flexible, capable of generating options and alternatives; effectively integrate technology push with business pull; address in a co-ordinated manner the whole range of activities from the holistic strategic level down to relatively small details; directly address the need to secure buy-in and involvement at all levels, with commitment to implement the outcomes" [Barker, Smith, 1995]. Roadmapping has particular strengths, of especial relevance in the context of this paper, including its potential to help navigate the "the multidimensional characteristics and complex nature of foresight studies" [Saritas, Oner, 2004]. In particular, roadmapping has the ability "to capture, manipulate and
1 Including Delphi, bibliometrics, stakeholder mapping, scenario planning, horizon scanning, expert panels, SWOT, citizen panels, etc. [Popper, 2008].
manage information to decrease complexity in the foresight by constructing roadmaps" [Saritas, Oner, 2004]. Because of the firm-level origins of technology roadmapping, the approach puts an "emphasis on visual, easy-to-comprehend descriptions of customer needs, technology responses, and R&D programmes offers several benefits" [Barker, Smith, 1995]. Roadmapping has, therefore, qualities that lend themselves to generating evidence that is strategically relevant and usable by R&D organizations. As highlighted in the context of firmlevel roadmapping, the technique can effectively "facilitate communication at the operational commercial and technical level, and with senior management too, as well as providing a practical means for ensuring R&D programmes are apposite, correctly prioritized, and adequately resourced." [Barker, Smith, 1995].
Roadmapping's visual approach and systems perspective has a number of advantages. "The value of the graphical models is that they show R&D projects and requirements in context rather than in isolation, they can depict new perspectives rapidly and they can serve as a focal point for enhanced communications and more detailed total systems analyses" [Saritas, Oner, 2004]. Roadmapping's effectiveness at systems analyses is further enhanced by its ability to operate at a wide range of system levels (from company division level to firm-level to global industry-level) and address a range of innovation system phenomena (from market trends to emerging scientific R&D domains) [Kappel, 2001; Phaal, Muller, 2009]. This scalability and adaptability offers the potential to readily respond to the need for evidence in greater sub-system detail.
The Roadmapping Framework
In this section we explore key features of the architecture of the roadmapping framework. We review efforts to use the roadmapping framework canvas as a research tool to study emerging technology innovation dynamics and sociotechni-cal change. We conclude by summarizing the distinctive features of the roadmapping canvas which offer the potential for its application as a diagnostic tool to monitor and regulate the sufficiency, efficacy, and credibility of strategic foresight evidence as it is gathered.
To understand the distinctive features of technology road-mapping - and their potential to support the generation of foresight outputs that meet STI policy users' strategic evidence requirements - it is important to distinguish between the roadmapping framework, roadmap content, and road-mapping process (Table 1).
Phaal and Muller highlight the importance of distinguishing between the roadmapping framework (canvas) and the content (information, stakeholder perspectives, insights, etc) captured and organized within the framework [Phaal, Muller, 2009]:
1. An underlying information-based structure (the roadmap architecture) — how the information contained within the roadmap is organized, which represents the key elements of the system (layers and sub-layers of the roadmap), set against time.
2. An overlaying graphical layer, with format, style, and color chosen to represent the roadmap structure and its content for communication purposes. The multi-layered time-based format is posited as the most comprehensive and flexible format for developing roadmaps, although different graphical styles have been developed for summary and communication purposes."
These graphical roadmapping frameworks [Park et al., 2020; Phaal et al., 2004b] can be considered dynamic systems frameworks, with the architecture of the roadmap providing a coherent and holistic structure (and common language) within which the innovation pathways and the evolution of the system and its components can be explored, mapped and communicated [Phaal, Muller, 2009).
In its most generic form, the visual roadmap is a time-based chart, comprising a number of layers, corresponding to a range of different innovation activities, typically including commercial and R&D perspectives (Figure 1). The roadmap enables the evolution of markets, products (and services), and the innovation pathways of technologies to be explored, together with the linkages, interdependencies, and discontinuities between the various perspectives. The roadmapping approach draws together key concepts from the technology strategy and transitions literature, by the use of its layered innovation activity structure set against the time dimension [Phaal et al., 2004a].
Within the generic roadmap canvas, three broad layers are set horizontally across the two-dimensional space of the canvas with a horizontal time axis [Phaal et al., 2004b; Phaal, Muller, 2009], corresponding to:
• A top 'purpose' layer, capturing 'know-why' innovation information: This layer captures evidence and insights related to trends and drivers that govern the overall goals or purpose associated with the roadmapping activity.
• A middle 'delivery' layer, capturing 'know-what' information: This layer captures evidence relating to the tangible systems that need to be developed to address strategic opportunities and challenges, and respond to trends and drivers (captured in the top layer). In firm-level road-maps, this typically corresponds directly to the evolution of products in terms of their functions, features, and performance. In innovation system-level roadmaps, this often corresponds to the functions, features, and performance of technology platforms (upon which private sector applications, products, and services are based).
• A bottom 'resources' layer, capturing 'know-how' information: This layer captures evidence related to the resources that need to be marshalled to develop the required products, services and systems, including knowledge-based resources, such as technology, skills, and competences, but also other resources such as finance, partnerships, and facilities.
The process by which this evidence is gathered, integrated, and synthesized is outlined in the following section.
The Roadmapping Process: Phases and Activities
The accumulation of foresight evidence within the roadmap-ping canvas typically happens within a sequence of steps or phases. The transitions from one stage to the next offer opportunities to reflect on the data gathered, the emerging patterns, and evidence gaps. In particular, these are opportunities to adapt the focus and granularity of analyses, reallocate resources and effort, and introduce new stakeholder perspectives and expertise, as appropriate.
Depending on the scope and ambition of the exercise, the stages of a roadmapping study can take place within a single event involving a single group of stakeholders or can be part of a staggered set of exercises, integrating information from other analyses, and inviting participants with different perspectives and expertise at different stages. The sequencing of
able 1. Roadmapping Dimensions
Item Description
Framework Dimensions, elements, organizing principles and graphical canvas within which evidence and strategic information is gathered
Content The evidence - data, insights, perspectives, and so on, as well as strategic priorities, milestone, and goals, which are collected and organized within the framework
Process The stages, activities and sequencing of actions related to collecting and organizing foresight evidence and strategic/planning information
Source: authors.
evidence-gathering phases - from more exploratory analysis (of trends and drivers, opportunities and challenges) to increasingly more strategically-focused considerations - is discussed in more detail below.
While there are a range of different approaches to defining and organizing the different phases and activities of a road-mapping process [Bray, Garcia, 1997; Nimmo, 2013; Phaal et al., 2007; Yasunaga et al., 2009], most approaches involve some version of the following: establishing a vision; exploring the landscape of capabilities and opportunities; and revealing innovation pathway options. In practice, these steps are often preceded by a preliminary planning phase - clarifying road-map aims and are followed by implementation and evaluation phases: translating and implementing outputs; validating and re-visiting the roadmap (Table 2).
This phased approach to gathering, integrating, and synthesizing foresight evidence is an important aspect of what makes the roadmapping process so adaptable - offering opportunities to reconfigure foresight efforts and resources to ensure the outputs are as useful and credible as possible.
Distinctive Features of Roadmapping (Supporting Evidence Diagnostics)
In this section, we highlight key aspects of roadmapping which support its functionality as an evidence diagnostic
tool - allowing the managers of foresight exercises to examine whether the evidence and insights gathered have sufficient granularity, relevant innovation focus, and stakeholder credibility. In particular, we review roadmapping's visual, integrating, scalable, iterative, and systemic nature and reflect upon the implications for the monitoring and regulation of foresight evidence collection.
The Visual Nature of Roadmapping
The graphical technology roadmapping canvas is designed to reveal patterns in evidence related to the complex innovation system dynamics of emerging technologies. In particular, the visual nature of the roadmapping approach means that the roadmapping-based tool can more effectively reveal temporal relationships between key events in different innovation activity domains [Park et al., 2020]. Saritas and Oner characterize the roadmapping methodology in terms of its ability to "capture, visualize, manipulate and manage information to decrease complexity in foresight" [Saritas, Oner, 2004].
Roadmapping's visual representation of "customer needs, technology responses, and R&D programmes" supports dialogue and communication between stakeholders from different operational, commercial, and technical perspectives, but also supports implementation, offering "a practical means for ensuring R&D programmes are apposite, correctly prioritized, and adequately resourced" [Barker, Smith, 1995]. The visual nature of the roadmapping framework not only helps reveal patterns in evidence, but also potential gaps in evidence, the paucity of detail in particular areas of the innovation canvas, and unexplained linkages or correlations deserving further attention. These opportunities to monitor and regulate evidence gathering, based on observed patterns, is discussed in more detail below.
The Integrating Nature of Roadmapping
The roadmapping framework is designed to gather and integrate evidence and insights from a range of innovation system stakeholder perspectives related to system activities, linkages, and elements, at different stages of innovation lifecycles. Furthermore, the canvas can be used to capture exploratory evidence (scanning future trends, opportunities, challenges),
Figure 1. Basic Multi-Layered Roadmapping Architecture
Typical viewpoints
Commercial y( and strategic perspectives ^
Design, development and production perspectives
Technology and research perspectives
Perspectives
(roadmap architecture)
4
Roadmap framework
(supports integrated and aligned strategic and innovation planning) |Time'
Medium-term Long-term Vision
Knowledge types
When?
Why?
What?
How?
Information
types
Drivers Strategy Needs
Form
Function
Performance
Solutions
Capabilities
Resources
Three key questions
2) Where are 3) How can we now? we get there?
1) Where do we want to go?
Source: [Phaal, Muller, 2009].
Table 2. Roadmapping Stages
Stage Contents
Clarifying roadmap aims Articulating and specifying the roadmapping exercise's focus and scope as well as intended outputs and impact
Establishing a vision Scanning trends, opportunities/challenges and establishing a consensus vision among participants
Exploring the landscape of capabilities and opportunities Surveying current (and potential) capabilities and opportunities; evaluating the relevance of particular capabilities to address opportunities; identifying innovation barriers and requirement gaps; and prioritizing innovation opportunity destinations
Revealing innovation pathway options Investigating strategic innovation pathway options - navigating innovation barriers, filling requirements gaps,leveraging enabling factors, and identifying potential intermediate goals and milestones
Translating and implementing outputs Translating roadmapping outputs into the strategic planning exercises of stakeholder groups and organizations (including informing technology selection and investment processes)
Validating and re-visiting the roadmap Following up with exercises to critique and validate elements of the roadmap, updating trend data and stakeholder insights, revising strategic goals and milestones
Source: authors.
normative forecasting evidence (focusing on desired future innovation system states), as well as strategic information (potential planning milestones and strategic goals) - all within a single integrating system framework. While the roadmapping framework is often used to gather input from stakeholders within facilitated workshops, the roadmap canvas can also be (pre-) populated with data and insights gathered from other foresight and strategy development processes2 [Hussain et al., 2017; Kanama et al., 2008; Oliveira, Fleury, 2015; Saritas, Oner, 2004; Strauss, Radnor, 2004; Vishnevskiy et al., 2015]. The juxtaposition on a single integrating canvas of evidence from different sources, system levels, and expertise facilitates efforts to reveal mismatches in stakeholder perspectives and potential gaps in evidence and sampling.
The Scalable Nature of Roadmapping
Not only can individual roadmapping exercises be carried out over a range of different scales and time periods but, in principle, the adaptive nature of roadmapping allows analyses which 'zoom in' and 'zoom out' to explore micro- or macrosystem dynamics which are deemed important by roadmap-ping participants. Roadmapping can operate at different system levels, with most roadmaps positioned at either the industry-level (or innovation system level) or firm-level, with some variations in emphases [Kappel, 2001] in terms of focusing on sector trends and market failures (roles for government) to intra-firm coordination issues and product market opportunities. As highlighted by Phaal and Muller:
"Roadmaps can cover a tremendous 'dynamic range', in terms of scale and complexity of the system. For example... a sector roadmap can be viewed at the level of a limited set of sector trends (order of magnitude 101)... or a complex system (within a sector) determined by millions of details (order of magnitude 107)... The scientific foundation of the technologies used in the systems may be orders of magnitude more detailed again. Nevertheless, the purpose of a roadmap is often to align scientific efforts with the sector trends. Roadmaps provide a means for addressing this complexity." [Phaal, Muller, 2009]. As discussed above, however, roadmapping has the potential to zoom in/out to investigate particular innovation activities/ dynamics within a roadmap. The architecture of the roadmap-ping framework can be "configured to suit the focus and scope of the issue being addressed" [Phaal, Muller, 2009]. The scalability
of roadmapping - its potential to "'magnify' and focus on the issues and areas of the system of most importance" - facilitates efforts to reveal inadequacies in the granularity of evidence being gathered or the focus on key innovation subsystems.
The Iterative Nature of Roadmapping
As discussed above, the different stages within the roadmapping process offer opportunities to reflect on the data gathered, on the emerging data patterns and evidence gaps. In particular, these offer opportunities to adapt the focus and granularity of analyses, reallocate resources and effort, and introduce new stakeholder perspectives and expertise, as appropriate.
A particularly important aspect of the iterative nature of roadmapping is the opportunity to engage new participants with important perspectives and expertise. As pointed out by Phaal and Muller, the roadmapping process "is somewhat paradoxical in that the appropriate expertise must be employed to develop a roadmap, but the appropriate expertise becomes fully known only after a complete roadmap has been constructed. An iterative roadmap development process is, therefore, essential" [Phaal, Muller, 2009]. In practice, therefore, many roadmaps are created in multiple iterations. A first iteration is often done in a short time-span, typically in one day (or a small number of days), often within a single workshop. Subsequent iterations may require more time - from a few days to a few weeks or months - depending on the complexity of the system being studied and one's ability to access relevant expertise as well as other contextual factors. These iterations ensure feedback between key perspectives (e.g. related to market trends and business opportunities; product, production, and operational requirements; technology and research capabilities). These iterations take place within an overall process of increasing focus during the roadmap creation, as participants converge on key elements of the roadmap based on the evidence gathered. Each iteration progresses through the same four phases of ideation, divergence, convergence, and synthesis [Phaal, Muller, 2009].
The ways in which the iterative nature of roadmapping might be used to more systematically identify opportunities to adapt the focus and granularity of analyses, reallocate resources and effort, and introduce new stakeholder perspectives and expertise, will be explored in more detail below.
2 e.g., SWOT, scenario planning, horizon scanning, Delphi.
The Systemic (and Multi-Perspective/Multi-Disciplinary) Nature of Roadmapping
Technology roadmapping frameworks are structured to explore different perspectives on a particular technology innovation journey from different parts of the innovation system: technology and research; design, development, and production; commercial and strategic [Phaal et al., 2004a; Phaal, Muller, 2009]. Furthermore, as highlighted by Saritas and Oner: "Today, most of the problems cannot be analyzed by a single discipline. All complex problems—especially social ones—involve a multiplicity of actors, various scientific/technical disciplines, various organizations and diverse individuals. In principle, each sees a problem differently and thus generates a distinct perspective on it" [Saritas, Oner, 2004]. As with foresight more generally [Georghiou, Keenan, 2006], an effective roadmapping exercise requires ensuring the involvement of a broad "range of actors engaged in science and innovation policy". From a practical perspective, advice on roadmapping processes often highlights the "importance of a suitable starting set of participating stakeholders" involving a breadth of innovation system perspectives in terms of "including technological + economic social or political aspects" [Schuh et al., 2013]. Similarly, guidance on roadmapping highlights the need to avoid "being isolated" pointing out that "roadmap building projects usually depend on a high degree of interdisciplinarity", and the importance of avoiding a "lack of coherence", emphasizing the importance of relating "issues of major interests (e.g. technologies) to other issues relevant in that context (e.g. products, applications and/or developments in the political, economic, social environment)" [Isenmann, 2008]. Examples of high-profile roadmaps, e.g., the US Department of Energy roadmaps related to the building and construction sector, highlight the importance of involving stakeholders from across the innovation system - "participants representing all phases of building process/stakeholders (manufacturers, developers, contractors, owners, architects, engineers...)".
A key aspect of the systemic nature of roadmapping is its focus on exploring potential innovation pathways (and path dependencies) within the innovation system. High profile NASA technology roadmaps are defined in terms of their exploration of "needed technology candidates and development pathways" [NRC, 2012]. The ability of roadmaps to offer evidence related to potential innovation pathways is critical to their appeal as a foresight tool that can inform strategy and planning. As highlighted by Saritas and Oner, however, "the pathways between research and eventual applications ('practical use' and 'widespread use' in UK foresight) are many, not necessarily linear and require an enormous amount of data for any attempt to link research with application. Substantial time and effort are required to portray these links as accurately as possible" [Saritas, Oner, 2004].
The ways in which the innovation system framing of the road-mapping canvas - and the focus on generating evidence on potential innovation pathways - could be used to ensure the relevance of outputs for roadmap users, in particular for STI strategy development and planning, will be explored in more detail below.
Lessons from Roadmapping as a Research Tool: Exploring Technology Innovation Pathways and Socio-Technical Transitions
As well as a practical foresight tool, the roadmapping canvas can also be configured as a research tool and used to capture
key innovation events and activities within studies of technological change, retrospectively and longitudinally [Phaal et al., 2007]. In particular, in this context, the roadmapping canvas has demonstrated its potential to distinguish, display, and scrutinize different categories and sources of empirical data. In this section, we briefly review some of the theoretical foundations and recent experiments in using roadmapping as a tool for studying technological change, and reflect on the implications for roadmapping practice and opportunities to enhance the relevance, granularity, and credibility of strategic evidence outputs.
In recent years, a number of researchers have used the road-mapping framework as an instrument to study emerging technology innovation trajectories and socio-technical transitions [Featherston et al., 2016; Featherston, O'Sullivan, 2017; Ho, O'Sullivan, 2019]. In particular, the roadmapping framework allows researchers to gather evidence in a structured way that follows an innovation system logic, helping to reveal linkages between key innovation system elements, actors, and activities (functions). In this context, the researchers have taken advantage of the correspondence between roadmapping dimensional layers and innovation system functions [Hirose et al., 2015; Ho, O'Sullivan, 2019; Park et al., 2020]. These research studies involved gathering and representing a variety of categories of evidence about key innovation events and activities influencing emerging technological innovation trajectories within the roadmapping framework. Rather than using workshops, the researchers collected information and insights from a variety of sources: semi-structured interviews, reviews of literature from technology and industry studies, standard databases, market analyst reports, and reviews of 'grey' literature (e.g., studies by government agencies or national academies, many of which draw upon an analysis of patent databases, bibliometrics, etc) [Park et al., 2020]. In methodological terms, it is worth noting that the use of the roadmapping framework as a research tool has some correspondence with other approaches to studying technological change. For example, Van de Ven's [Van de Ven, 1993] use of a framework of 'event tracks' to study the emergence of industrial infrastructure that facilitates the transformation of scientific knowledge into technology-based products or services. These tracks are analogous to key categories of roadmapping layers and are used to explore how events and activities related to distinct categories of infrastructure (e.g., different institutional arrangements, resource endowments, and proprietary activities) co-evolve with technological innovation [Park et al., 2020]. The roadmapping framework also has some correspondence with the 'Multi-Level Perspective' framework deployed by, for example, for qualitative longitudinal case studies of technological emergence [Geels, 2002]. It allows for the systematic tracking of key transitions from niche to regime, paying attention to dimensions influencing technological transitions (e.g. sectoral policy, infrastructure, user practices, techno-scientific knowledge) [Park et al., 2020]. In this context, the research reflects the motivation of microtechnical studies exploring 'technological trajectories' associated with specific 'technological paradigms' [Dosi, 1982]. For such studies it is critical to:
"identify with sufficient precision the "dimensions" which characterize each broad technological paradigm and differentiate it from others. define the 'difficult puzzles' and unsolved difficulties of a technology which are often a necessary (although not sufficient) condition for the search for other ones. describe the transition from one technological path to another and assess
the factors which allow the emergence of a 'winning' technology" [Dosi, 1982].
These roadmapping framework-based studies of technological change [Featherston et al., 2016; Featherston, O'Sullivan, 2017; Ho, O'Sullivan, 2019; Park et al., 2020; Phaal et al., 2004b] have identified a number of potentially transferrable methodological lessons, effective practices and sources of evidence, some of which are part of existing roadmapping foresight processes, including:
Integrating and comparing evidence from a variety of sources: Complementing evidence gathered through workshops with national statistics, data from industry analyses, or studies by government organizations, national academies, etc. These studies often contain (semi-) quantitative data of various types, including the analysis of patent data, bibliometrics, market data, and relevant national economic accounts data. The graphical nature of roadmapping allows for the effective comparison of quantitative evidence (e.g. from market analyses, standards databases, patent databases, etc) with qualitative information about key events (and vice versa). In particular, it is possible to graphically overlay trend data with qualitative data points on a roadmap canvas and explore potential correlations (or inconsistencies) between, for example, key scientific, technology demonstration or business events with any inflection points in bibliometric, patent, or market data.
Characterizing and sampling innovation system stakeholder perspectives: When gathering evidence and insights from literature and archival sources, the roadmapping 'layers' (corresponding to categories of industrial-innovation functional activity) can also be used to characterize the perspective of the stakeholders providing the data3 as well as their national innovation system and sectoral innovation system contexts. This offers a systematic way of monitoring the sampling and consistency of stakeholder perspectives and inputs. In this context, the roadmapping functional perspective categories and innovation system boundaries enable researchers to compare (and potentially reconcile) different perspectives around the importance, interdependencies, and impact of particular innovation events and activities, barriers and enablers. 'Zooming-in' on key innovation events, barriers, linkages: The scalable roadmapping-based frameworks proved highly effective in supporting researchers to 'zoom in' on important micro-technical details which may have influenced the path dependencies of technology innovation journeys. In particular, where appropriate, researchers were able to study key roadmap features with greater granularity, introducing new roadmapping sublayers which distinguished, for example, between different types of technology (e.g. product, measurement tools and systems technologies); between varieties of R&D activities; and between different categories of institution (e.g. types of standards and regulations). Furthermore, the visual nature of the roadmapping approach means that the roadmapping-based tool can more effectively reveal temporal relationships between key events in different innovation activity domains. [Park et al., 2020].
Assessing evidence adequacy: While foresight evidence patterns within the roadmapping canvas helped identify potential requirements for further analysis (greater granularity, more careful analysis of innovation linkages, or finer stakeholder sampling, etc.), judgements needed to be made about the added value of more granular evidence on particular fea-
tures. In this context, researchers typically developed tests for the adequacy of evidence gathered for particular regions or features of the roadmap canvas. In particular, if the identification of new key roadmap features (events, barriers, linkages) 'saturates' - i.e. no new features deemed influential to the innovation dynamics being studied are being added to the mapping canvas, then investment in further iterations of analysis may not be justified.
In summary, the use of the roadmapping canvas as a research tool to study technology trajectories has highlighted the importance of obtaining sufficient granularity of evidence and triangulation of stakeholder perspectives to identify and understand the key factors influencing innovation path dependencies. In particular, these research studies point to the potential of the roadmapping framework as a diagnostic tool to examine the sufficiency, efficacy, and credibility of evidence related to technology innovation pathways.
Discussion: Foresight Evidence Patterns and Agile Roadmapping
In this section, we revisit the key foresight evaluation principles and evidence challenges within the visual organization of the roadmapping canvas. In particular, we investigate the potential of roadmapping frameworks to more effectively monitor and regulate the collection of strategic foresight evidence. We explore how some of the distinctive features of the roadmapping framework offer enhanced opportunities to address the challenges of ensuring the relevance and usability of foresight analyses; and to enhance granularity, coverage, and consensus of roadmapping data.
We start this section by summarizing roadmapping practices, outlining typical approaches to gathering inputs from stakeholders, capturing evidence and insights, and representing these graphically within the roadmapping canvas. We go on to explore data patterns associated with the foresight 'sources of error'. In particular, we examine patterns of evidence (related to particular innovation activities or dynamics) that may require more granular detail, broader stakeholder input, more attention to particular innovation system elements, or lifecycle phases. We conclude by highlighting how the iterative structure of roadmapping processes offers decision points when the overall strategy foresight exercise could be adapted and reconfigured (in terms of scope, focus, and prioritization of effort/resources) to ensure greater accuracy, credibility and utility.
Situations for Applying Enhanced Methods for Gathering Roadmapping Evidence, Insights, and Priorities
In practice, most roadmapping exercises involve facilitated workshops, in addition to other activities, where participants contribute information, priorities, and insights, which are collected and organized within the roadmapping architecture. A common approach is to use large wall charts (paper or digital), structured using the roadmap architecture, as the focus for the foresight activities [Phaal et al., 2007]. Sticky notes are often used as a mechanism for participants to contribute insights and evidence (Figure 2). As part of a facilitated process, participants identify specific innovation events, barriers, and opportunities they believe are important. They are also encouraged to articulate more general insights and perspectives they believe are relevant to the innovation dynamics being studied in the workshop.
3 e.g. scientists, technology engineers, manufacturing engineers, economists, market analysts, policy researchers, and analysts, etc.
Figure 2. Domain Landscape
Domain Landmarks 'Deep dive' into landmark
landscape in landscape (exploratory roadmap)
Synthesis to create domain roadmap
Note: (a) Photograph of domain landscape; (b) arrows highlighting important 'landmarks'; (c) coloured dot stickers capturing different stakeholders' prioritization; (d) 'deep dive' exploration of innovation dynamics related to particular roadmap 'landmarks'
Source: authors.
The roadmap layers (and sublayers) are used as a checklist to stimulate the generation of ideas. Inputs can include a wide variety of information types, from important innovation trends and drivers, strategic opportunity options, key innovation activities to barriers and risks, speculation and scenarios; as well as strategic planning information, milestones, and goals. Different types of information are typically added at different phases of the roadmapping foresight process. These different stages are discussed below.
As described by [Phaal et al., 2007], the key metric used in populating the roadmapping canvas is the density and distribution of sticky notes. When participants identify links between roadmap content then these are often captured by drawing connecting arrows. The contributions of individual participants can be analyzed to reveal clusters of similar features and ideas (Figure 2b), before duplicates are removed (or rationalized) and key events and/or opportunities are collectively prioritized. A common workshop practice for prioritizing opportunities is to use a 'sticker vote' technique, where participants vote on the importance of key roadmap features by adding small colored 'dot' stickers adjacent to the relevant opportunity or event on the roadmap canvas (Figure 2c). The votes are counted by the facilitators to identify the most interesting roadmapping opportunities within a 'domain landscape'. The roadmapping exercise may then involve a number of 'deep dive' analyses of particular roadmapping features or 'landmarks' (Figure 1d). The insights from this analysis are then synthesized with information from other deep dives to enrich the information captured within the overall roadmap-ping canvas.
The roadmapping practices described above illustrate how the focus of a roadmapping exercise can be adapted to investigate key aspects of the emerging understanding of innovation pathways. In particular, it is possible to explore particular innovation activities, linkages, and dynamics in greater detail to distinguish between different stakeholder perspectives and priorities, and iteratively synthesize the new insights into an increasingly richly populated roadmapping canvas. These benefits can be rapidly realized even for large-scale foresight initiatives, in a one- or two-day workshop with carefully selected participants, as a first (often design) iteration for a more substantial foresight initiative, or as a one-off diagnostic. In the following sections, we explore how some of the distinctive scalability and systemic features of the roadmapping
framework offer enhanced opportunities for the monitoring and regulation of foresight evidence collection, addressing the challenges of ensuring the relevance and usability of foresight analyses and the potential to enhance granularity, coverage, and consensus of roadmapping data.
Event Distribution Patterns: The Density (or Absence) of Stakeholder Inputs Within Particular Regions of the Roadmapping Canvas
As discussed above, key metrics used when considering the evidence population of a roadmapping canvas are the density, connections, and distribution of sticky notes. Figure 3 schematically illustrates how roadmapping evidence can be concentrated within a particular zone of the roadmap's innovation canvas. This high density clustering of information (within adjacent layers related to particular sets of innovation activities, within a particular innovation time window) signals a convergence of attention by participants on a zone of innovation dynamics perceived as important to determining the technology trajectory and innovation pathways. A high density of stakeholder inputs in a particular region of the roadmapping canvas may signal an important area of innovation dynamics, which merits more careful and detailed consideration. Such clusters should be scrutinized within the facilitated workshop and potentially examined in greater detail in a further iteration of the analysis - zooming in on the region of interest with more 'granular' roadmapping layers corresponding to more specific categories of innovation activity. In scrutinizing clusters of evidence within the roadmapping canvas, it may also be important to analyze the level of consensus and sampling of innovation system stakeholder perspectives. For example, a set of participant inputs clustered within the roadmapping canvas may be coming from only one specific stakeholder group. This in turn may reflect the insights and expertise of that particular group, or it may reflect a lack of consensus or variation in priorities. As illustrated in Figure 4, it may be important to distinguish between and analyze the sampling of stakeholder perspectives. Significant levels of variance between stakeholder perspectives may need to be reconciled, either within a facilitated workshop setting or by examining the issue in a further iteration of the analysis involving appropriately augmented stakeholder groups. Similarly, it may be important to capture and scrutinize stakeholder inputs both in terms of the level of confidence they
Figure 3. Density of Roadmapping Objects (Events, Barriers, etc)
Sticky notes representing insights and evidence contributed by roadmapping participant (e.g. innovation events, barriers, opportunities)
High 'density' of evidence within particular zone of the innovation canvas', indicating potential need to 'zoom in
Source: authors.
have in issues or features they have highlighted, as well as their expertise in the relevant innovation activities. As schematically illustrated in Figure 5, clusters of inputs may vary in terms of the underpinning confidence levels of the participants or the relevance of their expertise. Clusters of evidence with significant numbers of inputs with low confidence or low expertise may need to be examined further to diagnose lack of confidence in an input (e.g. related to the probability of an innovation event happening or the importance of an innovation activity). Clusters of inputs by participants in areas of perceived importance, but where there is limited expertise, may need to be examined in a further iteration of the analysis involving participants with relevant specialities. Many clusters of evidence occur within particular 'zones' of the roadmapping canvas, associated with adjacent layers on the framework and a particular period of innovation activity. The relative positioning of particular roadmapping layers within the architecture reflects a general sequencing of innovation activities from technology R&D to product and market development. Interactions and feedback loops can, of course, occur between the layers - reflecting the non-linear nature of the innovation process. In this context, not all important evidence patterns will necessarily reside in adjoining road-map layers. The importance of identifying patterns involving linkages between roadmapping evidence (innovation events, activities, barriers, etc) is highlighted in the following section.
Linkage Patterns: The Spread and Span of Roadmapping Linkages across the Roadmapping Canvas
As discussed above, roadmapping participants are encouraged to highlight important linkages between roadmapping landmarks, including linkages between features within non-adjacent regions of the roadmapping canvas. There are a number of roadmapping evidence linkage patterns which merit attention. In particular, some roadmapping landmarks (especially those related to catalytic innovation events or rate limiting innovation activities) can be linked to a multiplicity of other features on the roadmapping canvas. This density and spread of linkages is schematically illustrated in Figure 6.
Roadmapping landmarks with a high number of linkages to a range of other roadmap layers and objects may need to be analyzed further, potentially by examining the issue in a further iteration of the analysis involving additional expertise associated with the innovation activities which are being linked to from the landmark roadmap feature.
As discussed above, the relative positioning of particular roadmapping layers within the architecture reflects a general sequencing of innovation activities from technology R&D to product and market development. Linkages between innovation features which are spatially separated on the roadmap-ping canvas are, in reality, generally mediated by some or all
Figure 4. Density of Perceived Importance of Innovation Roadmap 'Object' by a Single Stakeholder Group Perspective
Sticky notes representing insights and evidence contributed by roadmapping participants, colour-coded by stakeholder group
'Density' of stakeholder evidence,
indicating potential importance or complexity to a particular group
Source: authors.
Figure 5. Variations in Stakeholder Confidence/Expertise in an Object They Have Contributed to the Roadmap
u
Sticky notes representing insights and evidence contributed by roadmapping participant (e.g. innovation events, barriers, opportunities)
Source: authors.
High confidence/ J
high expertise
High confidence/ low expertise
Low confidence/ high expertise
Low confidence/ low expertise
U 9
Evidence contributions coded for level of confidence and expertise of the contributing foresight participant
of the intervening innovation activity layers. In this context, 'long' linkages identified by roadmapping participants - i.e., those without 'stepping stone' connections through intermediating innovation activity layers - may indicate the need to further analyze the intermediating innovation pathways to identify any barriers along the way. A schematic illustration of a long linkage between innovation objects on the roadmap canvas is represented in Figure 7.
In this paper, we have focused in particular on the importance of generating roadmapping foresight evidence to support the development of strategies for emerging technologies. Under these circumstances, the roadmapping evidence needs to help reveal information on potential technology innovation pathways - from technology R&D, application demonstration, product design, industrialization, and business model/ market development.
In this context, the roadmapping canvas can also be used to capture strategic information from participants. In particular, the roadmap can be used to capture potential milestones and intermediate strategic goals. It can also highlight potential innovation pathways linking them. A schematic illustration of the series of linkages indicating a potential innovation pathway from technology proof of concept to a final strategic opportunity goal is captured in Figure 8.
The evidence patterns described above are all signals that particular innovation activities and dynamics may need to be studied more carefully. In particular, as the understanding of potential innovation pathways becomes clearer, there may be a need to adapt the roadmapping exercise to scrutinize particular clusters of evidence, linkages, and stakeholder consensus.
Evidence Patterns and Roadmapping Process Adaptation
In this section, we explore how the different phases and stages of the roadmapping process offer the opportunity to re-scope, re-prioritize, and re-focus foresight efforts and resources to enhance the granularity, coverage, consensus, and strategic relevance of the evidence generated.
As discussed above, the collection, integration, and synthesis of foresight evidence within the roadmapping canvas typically happens within a sequence of steps or phases, including:
• Establishing a vision: Scanning trends, opportunities/ challenges and establishing a consensus vision among participants;
• Exploring a 'landscape' of capabilities and opportunities: surveying current (and potential) capabilities and opportunities; evaluating the relevance of particular capabilities to address opportunities; identifying innova-
Figure 6. Multiplicity of Interdependencies/Linkages from a Particular Roadmapping Object to a Range of Other Dispersed Innovation System Activities
Evidence contributed by roadmapping participant (e.g. innovation events, barriers, opportunities)
'Density' of evidence interdependences: Multiple linkages to/from a particular innovation 'event' (or activity, barrier, etc.) to other evidence
Source: authors.
Figure 7. Differences in the 'Distance' between Linked Innovation 'Objects' on the Roadmap Canvas
Evidence contributed by roadmapping participant (e.g. innovation events, barriers, opportunities)
Innovation 'distance': Extent of separation between linked evidence 'objects' (i.e. without 'stepping stones' through intermediating innovation activity layers/dimensions)
Source: authors.
Figure 8. Series of Linkages Indicating a Potential Innovation Pathway from Technology Proof of Concept to a Final Strategic Opportunity Goal
Strategic opportunity goal
Technology
proof of concept
Sticky notes representing insights and evidence contributed by roadmapping participant (e.g. innovation events, barriers, opportunities)
Innovation pathway illustrating potential sequence of particular innovation activities and events from proof of concept to opportunity goal
Influence arrow highlighting additional key innovation events and activities shaping the innovation pathway
Source: authors.
Figure 9. Variations in Evidence Emphasis within the Roadmap Canvas for Different Phases of Foresight Analysis
<1 -—ll. •
» •
• •
- Time
•
a n ~
! u % pj
t'"
(a) Horizon scanning:
SCANNING for opportunities/ challenges
(b) Landscaping:
MATCHING and prioritising opportunities and capabilities
(c) Roadmapping:
MAPPING opportunity 'pathways', exploring barriers and enablers
(d) Strategising:
PLANNING for R&I programme along
specific 'pathway' (with milestones, actions, goals)
/v Potential 'innovation pathway' • Innovation activity or event
Q Innovation barrier or inhibitor Innovation catalyst or enabler
Ongoing accumulation of knowledge ^ Focusing on meaning / purpose ^ Convergence of strategy evidence
Source: authors.
Table 3. Main Focus Areas of Roadmapping
Phase Contents
1. Establishing a vision The roadmapping participant inputs (dot votes) are focused on longer term potential opportunities and future innovation system features, systems, and technology breakthroughs
2. Exploring the landscape The roadmapping participant contributions are focused on surveying current (and potential) capabilities and relevant future opportunities; evaluating the relevance (and strengths) of those capabilities to address the opportunities
3. Mapping potential innovation pathways The roadmapping participant contributions are focused on investigating strategic pathway options (navigating barriers, filling requirement gaps, identifying potential milestones, etc)
Source: authors.
tion barriers and requirement gaps; and prioritizing innovation opportunity 'destinations';
• Revealing innovation pathways: Investigating strategic innovation pathway options - navigating innovation barriers, filling requirements gaps, leveraging enabling factors, and identifying potential intermediate goals and milestones.
These stages are followed by a process of translating the foresight outputs into the strategic planning exercises of STI stakeholder organizations (including informing strategic technology selection and R&D investment processes). The population of the roadmapping canvas with evidence for the different phases of 'scanning, 'landscaping, 'roadmapping, and (pathway) 'planning' is illustrated schematically in Figure 9. The dots scattered on each canvas correspond to new inputs contributed by roadmapping participants in each phase, with the patterns reflecting the foresight emphasis at each stage.
Each roadmap canvas schematic is separated into three layers (corresponding to the dimensions discussed above). As be-
fore, the top layer captures evidence and insights related to trends and drivers relevant to the goals of the mapping activity; the middle layer captures evidence related to systems that need to be developed to address the opportunities; and the bottom layer captures inputs related to the enablers and resources (including science & technology research) (Table 3). Following the formal roadmapping foresight exercise, there will be a stakeholder planning phase (4), where the foresight outputs will be translated into STI organizations and used for their strategic planning purposes. In particular, the focus of these exercises will be on generating strategic inputs as part of selecting particular strategic pathways (including specific milestones, intermediate innovation 'stretch goals' and final strategic objective goals).
The iterative nature of the extended roadmapping exercise is illustrated in Figure 10, highlighting the ongoing interplay between exploratory, normative, and strategic analyses. This figure also identifies potential decision points between key stages of the analysis, offering opportunities to revisit the scope, focus, and priorities of the roadmapping exercise, and make any appropriate revisions to the allocation of resources and effort.
The figure also illustrates the push-pull dynamic between the initial scope and ambition of analysis set by commissioners of foresight exercises and the evidence requirements of innovation stakeholders (who will use the outputs to develop their strategies). This dynamic underpins the tension between the resources and efforts allocated to carry out a foresight exercise and the granularity, credibility, and efficacy of the evidence gathered. It is only as the roadmapping exercise progresses -and the landscape features and potential innovation pathways start to emerge - that it will start to become clearer which areas of the roadmapping canvas will require greater attention and whether the evidence requirements of future road-map users can be achieved. In this context, it is critical that the roadmapping process is adaptive, iterative, and agile. There are opportunities to re-focus, re-scope, and re-prioritize foresight efforts in response to the complexity of the innovation system dynamics being explored.
Figure 10. Iterative Nature of the Extended Roadmapping Exercise, Involving Ongoing Interplay between Exploratory, Normative, and Strategic Analyses
Structuring the foresight
exercise:
• terms of reference
• scope of study
• system boundaries (relevant stakeholder perspectives)
• allocation of resource/ effort
Strategic evidence requirements of users of the foresight outputs
Scanning
for future opportunities / challenges
Landscaping:
mathching / prioritising opportunities and capabilities
barriers / enablers; revealing opportunity 'pathways'
Output and impact:
• Absorption of evidence by stakeholders
• Application within stakeholdeer roadmap (milestones, actions, goals, etc.)
/V Potential 'innovation pathway' • Innovation activity or event Foresight decision points Revisiting the scope, focus, and
priorities of the analysis; and
o Innovation barrier or inhibitor * Innovation catalyst or enabler allocation of resources/effort
Source: authors.
Figure 11. Iterative Nature of the Extended Roadmapping Exercise, Involving an Ongoing Interplay between Scanning, Landscaping, and Mapping Analyses
Structuring foresight exercise:
Setting terms of reference, scope of study, system boundaries (relevant stakeholder perspectives), allocation of resource/effort
Source: authors.
•
/ Z'AJL-*'
In practice, roadmapping-based foresight efforts involve ongoing feedback loops between exploratory and normative analyses as well as strategic planning exercises. There can be any number of iterations involving roadmapping workshops (and integration and synthesis of evidence from complementary foresight analyses). Indeed, roadmaps may be updated by stakeholders at regular intervals beyond the lifetime of a particular foresight exercise or strategic planning lifecycle. This ongoing interplay between scanning, landscaping, and mapping analyses is illustrated in Figure 11. The iterative and adaptive nature of roadmapping processes, combined with a roadmapping framework which can facilitate the monitoring and regulation of evidence offers the potential to significantly enhance the granularity, efficacy, and credibility of foresight outputs. In the following section, we summarize the ideas discussed in the previous sections into a set of 'principles for adaptive roadmapping'.
Implications for Foresight Practice and Application: Principlesior Adaptive, Effective Roadmapping
There are significant challenges in ensuring the outputs of roadmapping foresight exercises meet the evidence requirements for technology strategy development in STI agencies and research & technology development (RTD) organizations. In many cases, the monitoring and regulation of evidence-gathering related to technology innovation pathways does not effectively adapt as critical elements of the complex innovation system dynamics are revealed. Without a systematic approach to adaptively re-scoping and re-focusing foresight resources and efforts, it is difficult to ensure the roadmap-ping outputs are strategically relevant, detailed, and credible for users. In order to reduce sources of 'evidence failure' and increase user impact, the following adaptive roadmapping principles are important:
Design the scope and focus of the roadmapping study in the context of foresight users' strategic evidence requirements:
For those roadmapping studies where the outputs are intended to inform strategy development at STI policy agencies and R&D organizations, it is important for those commissioning roadmapping studies to have early and ongoing engagement
with intended users. In particular, the scope and focus of the study should be structured to ensure the evidence/insights generated are focused on the right units of analysis and the right granularity of detail. If this is not the case, the transition from a foresight evidence base and analysis of potential innovation pathways into the strategy development process of an STI stakeholder (see Figure 9(c)-(d)) may not be effective. The dimensions and phases of the roadmapping framework canvas should be structured accordingly to ensure outputs in a format that is fit-for-purpose, absorbable by intended users, and can ensure the timelines considered are relevant in the context of the lifecycles of government STI policies, agency programs, and RTD organization planning.
Allocate sufficient time and resources to meet the granularity of evidence requirements (and revisit the distribution of resources and efforts as evidence is gathered): There is an inevitable push-pull dynamic between the scope of analysis set by commissioners of foresight exercises and the evidence requirements of innovation stakeholders using the outputs to develop strategies. This dynamic underpins the tension between the resources and effort allocated to a foresight exercise and the granularity/efficacy of evidence gathered. When foresight evidence patterns (within the roadmapping canvas) suggest a need for further analysis (see Figures 3-7), there may not be sufficient resources (or time) available within the originally designed foresight exercise. In this context, decisions will have to be made about potentially narrowing the scope of the analysis to achieve the required granularity and sampling of key issues, or further resources may need to be requested. Again, the patterns of evidence within the roadmapping canvas (Figures 3-7) may offer insights into how the exercise could be effectively re-bounded in scope (without compromising on accuracy in critical areas). In particular, earlier iterations of the scanning or landscaping analyses should reveal opportunities to reprioritize and refocus attention on particular innovation events and activities (which are critical to determining the trajectories of technology innovation pathways).
Take an adaptive and iterative approach to scoping, focusing, and sampling roadmapping foresight evidence: The gathering, integration, and synthesis of foresight evidence within a roadmapping canvas typically happens within a sequence of
steps or phases. In particular, roadmapping exercises generally involve phases of exploratory, normative, and strategic (options) analysis (see Figures 10 and 11). These phases offer potential decision points for foresight adaptation. In particular, there are opportunities within the extended foresight exercise, at feedback stages between scanning, landscaping, and mapping analyses to revisit the scope, focus, and priorities of the analysis, and make any appropriate revisions to the allocation of resources and effort (as discussed above).
Systematically scrutinize different categories of evidence patterns to identify areas requiring more careful analysis:
The visual nature of the roadmapping framework canvas (and its underpinning innovation system architecture) offers the opportunity to graphically reveal different categories of evidence patterns, which may signal a need for more detailed and careful analysis, in particular: (1) evidence cluster patterns, i.e. where there is a high density concentration of participant inputs (see Figures 3 and 4), signaling a convergence of attention on a zone of innovation dynamics, which may merit more careful and detailed consideration; (2) evidence linkage patterns, i.e. where there is a significant spread and span of roadmapping linkages from a particular landmark across the roadmapping canvas (Figures 6 and 7), signaling the identification of a roadmapping feature with potentially complex innovation system dependencies; and (3) stakeholder input patterns, i.e. where there is significant variance in the consensus, confidence, and sampling of innovation system stakeholders around innovation events and system linkages (See Figure 8).
Apply tests to determine evidence adequacy and relevance:
While foresight evidence patterns within the roadmapping canvas may identify a potential need for further analysis (greater granularity, more careful analysis of innovation linkages, or finer stakeholder sampling, etc.), a judgement will need to be made about the likely diminishing strategic returns of gathering additional evidence at each iteration. In this context, it will be important to develop tests for the adequacy of evidence at different phases of the analysis, in particular: (a) If further detail or finer sampling has not significantly changed the characterization of key events or the trajectories of key innovation pathways revealed in the roadmap, then investment in further iterations of analysis may not be justified. (b) If the extra detail from further iterations would go beyond the granularity of the strategy development needs of the users of the foresight outputs, then investment in further iterations of analysis may not be justified
Implications for Theory and Research Methods
In the previous sections, we explored the potential of the roadmapping canvas framework to effectively structure the collection, organization, and analysis of foresight evidence. In addition to the implications for STI foresight practice, we argue that the practice-based roadmapping architecture has the potential to offer a flexible, scalable framework for academic study of innovation system dynamics and technological change, as well as informing our understanding of the process of foresight itself.
In this paper we explored the extended foresight process (from the commissioning of a study to the development of a foresight-informed strategy) within the visual organization of the roadmapping canvas. The graphical representation of key distributions of evidence patterns at different phases of foresight offers some semantic clarity and precision in distin-
guishing exploratory, normative, and strategic analyses (and emphasizing their interdependence). In particular, we highlight distinct phases of foresight effort: scanning, landscaping, mapping, and planning (see description in Table 1). In representing foresight evidence within the roadmapping framework, we introduce the notion of the 'innovation pathway' as an important object of STI strategic foresight analysis - i.e., a continuous journey through the roadmapping canvas of different innovation system functions (Figure 8 and Figure 9(c)-(d)), starting with a particular STI-based capability and ending with a particular socioeconomic impact opportunity. This concept helps reveal what types of evidence are needed, how this evolves throughout the foresight process, and where there might be gaps. For example, if there is insufficient evidence to support an understanding of potential pathways, it will be more challenging for users to apply foresight evidence into strategy development. Without sufficient granularity to determine the path dependencies of the innovation trajectory, the evidence base may be inadequate. Similarly, without relevant innovation stakeholder perspectives informing the different stages of the entire innovation pathway, the evidence may not be credible.
More generally, the graphical representation of foresight evidence at different stages of the process highlights the inherent unknowns at the beginning of a foresight process and the importance of adaptation and iteration. At the commissioning stage of a foresight exercise, one cannot fully anticipate how complex the dynamics of a particular technology innovation pathway might be. One cannot, ab initio, know what level of microtechnical innovation detail may be required in order to determine potential innovation 'path [way] dependencies'. Similarly, one cannot anticipate the level of consensus or disagreement among innovation stakeholders regarding key events, trends, barriers, priorities, and so on influencing the innovation pathways. It becomes very clear that at the beginning of a foresight process, it will not always be possible to fully identify the right cohort of foresight participants (i.e. appropriate mix of perspectives and expertise) or strike the right balance of exercise scope and resources (in order to ensure outputs are sufficiently granular, focused, and credible to be actionable). Consequently, strategic foresight analysis of highly complex technology innovation systems must be both adaptive and iterative, if it is likely to ensure its outputs that are useful, trusted, and used.
Finally, we suggest that the specific patterns of evidence/information within the canvas highlighted above should also be useful in supporting academic research studies of technological emergence or socio-technical change. Again, the patterns (and associated principles) should help reveal where evidence on particular innovation activities or dynamics may require more granular detail, broader stakeholder input or more attention to particular innovation system elements or lifecycle phases.
Implications for Practice and Future Research
This paper explores the challenges of carrying out effective strategic technology roadmapping at the national innovation system-level or sector-level. We focus on roadmapping exercises designed to inform the STI strategies of policymakers and R&D agency officials (although the approach and findings may be transferable to firm-level or non-technology-focused foresight). In particular, we highlight the difficulties of ensuring that the outputs of foresight exercises have the right
level of detail, scope, and stakeholder confidence to support the strategy development needs of the STI policymakers and agency officials.
While the literature offers some insights on sources of error, evidence limitations, and how to evaluate foresight and road-maps, there is limited guidance on how to improve strategic foresight processes, performance, and impact. In particular, there is limited guidance on how to identify and mitigate deficiencies in foresight evidence granularity, relevance, and credibility as a foresight process is carried out and evolves. This paper reviewed the use of roadmapping as a foresight and strategy tool, highlighting distinctive features and functions by comparison with other foresight tools. We examined the application of roadmapping frameworks as an innovation research tool (to study emerging technology innovation dynamics and sociotechnical change). In particular, we highlighted how the distinctive features of the roadmapping canvas offers the potential for its application as a diagnostic tool to examine the sufficiency, efficacy, and credibility of strategic foresight evidence as it is gathered.
We argue that the distinctive features of roadmapping frameworks and processes offer the potential to more effectively monitor and regulate the collection of strategic foresight evidence. In particular, we highlight how the visual organization of roadmapping within an innovation systems perspective means that specific patterns of evidence within the canvas can signal where evidence on particular innovation activities or dynamics may require more granular detail, broader stakeholder input, more attention to particular innovation system elements, or lifecycle phases.
Furthermore, we outline how the iterative structure of road-mapping processes offers decision points where the overall strategy foresight exercise could be adapted and reconfigured (in terms of scope, focus, and prioritization of effort/ resources) to ensure greater accuracy, credibility, and utility. In particular, we highlight the potential for reflection points between phases of 'scanning' (for future opportunities/challenges); 'landscaping' (the layout of capabilities and opportunities); 'mapping' (potential strategic pathways); and 'strat-egizing' (for milestones, activities, and goals). In doing so
we offer some semantic precision (and hopefully clarity) for these common foresight terms.
Reflecting on the implications for roadmapping practice, we propose five 'principles for adaptive roadmapping':
1. Design the scope and focus of the roadmapping study in the context of foresight users' strategic evidence requirements
2. Design (and revisit) the allocation of time and resources to meet the granularity of evidence requirements
3. Take an adaptive and iterative approach to scoping, focusing, and sampling roadmapping foresight evidence
4. Systematically scrutinize different categories of evidence patterns to identify areas requiring more careful analysis
5. Apply tests to determine whether the evidence base is adequate (in terms of precision), sufficient (in terms of sampling), and relevant (in terms of user requirements and utility)
The principles proposed above are a preliminary set of methodological guidelines. Further work should test and refine these approaches in practice and identify other categories of evidence patterns for signals of evidence quality. Further work should investigate the potential to apply particular data analysis tools and methods (e.g., the Design System Matrix [Browning, 2001]), as well as opportunities offered by the application of digital tools to roadmapping exercises (capturing participant inputs in more structured ways; integrating data from other foresight analyses more systematically, etc). Further work should also investigate the implications of this work for other foresight methods, in particular the relevance, importance, and transferability of the 'agile' approach and principles.
Finally, the ability of the roadmapping canvas to effectively organize and support the analysis of foresight evidence suggests the fundamental roadmapping architecture may have the potential to offer a more general flexible, scalable framework for studying innovation system dynamics and technological change. This enables a better understanding of its utility (and limits), as well as any opportunities to enhance the architecture (or refine it for particular applications).
References
Baldi L. (1996) Industry roadmaps: The challenge of complexity. Microelectronic Engineering, 34(1), 9-26. https://doi.org/10.1016/S0167-9317(96)00013-5
Barker D., Smith D.J.H. (1995) Technology foresight using roadmaps. Long Range Planning, 28, 21-28. https://doi.org/10.1016/0024-6301(95)98586-H
Bray O.H., Garcia M.L. (1997) Technology roadmapping: The integration of strategic and technology planning for competitiveness. In: Innovation in Technology Management. The Key to Global Leadership (PICMET'97 Proceedings, Portland, OR, USA, 31-31 July 1997), pp. 25-28. https://doi.org/10.1109/PICMET.1997.653238 Browning T.R. (2001) Applying the design structure matrix to system decomposition and integration problems: A review and new
directions. IEEE Transactions on Engineering Management, 48(3), 292-306. https://doi.org/10.1109/17.946528 Cho Y., Yoon S.-P., Kim K.-S. (2016) An industrial technology roadmap for supporting public R&D planning. Technological Forecasting and
Social Change, 107, 1-12. https://doi.org/10.1016/j.techfore.2016.03.006 Coates V., Farooque M., Klavans R., Lapid K., Linstone H.A., Pistorius C., Porter A.L. (2001) On the Future of Technological Forecasting.
Technological Forecasting and Social Change, 67, 1-17. https://doi.org/10.1016/S0040-1625(00)00122-0 Day J. (2013) Review of cross-government horizon scanning: A Policy Paper for the UK Government Cabinet Office, London: UK Government. Dosi G. (1982) Technological paradigms and technological trajectories: A suggested interpretation of the determinants and directions of
technical change. Research Policy, 11(3), 147-162. https://doi.org/10.1016/0048-7333(82)90016-6 de Almeida M.F.L., de Moraes C.A.C., de Melo M.A.C. (2015) Technology Foresight on Emerging Technologies: Implications for a National Innovation Initiative in Brazil. Journal of Technology Management & Innovation, 10(2), 183-197. https://doi.org/10.4067/S0718-27242015000200013
Featherston C.R., Ho J.-Y., Brévignon-Dodin L., O'Sullivan E. (2016) Mediating and catalysing innovation: A framework for anticipating the
standardisation needs of emerging technologies. Technovation, 48-49, 25-40. https://doi.org/10.1016/j.technovation.2015.11.003 Featherston C.R., O'Sullivan E. (2017) Enabling technologies, lifecycle transitions, and industrial systems in technology foresight: Insights from advanced materials FTA. Technological Forecasting and Social Change, 115, 261-277. https://doi.org/10.1016/j.techfore.2016.06.025
Geels F. (2002) Technological transitions as evolutionary reconfiguration processes: a multi-level perspective and a case-study. Research
Policy, 31(8-9), 1257-1274. https://doi.org/10.1016/S0048-7333(02)00062-8 Georghiou L., Keenan M. (2006) Evaluation of national foresight activities: Assessing rationale, process and impact. Technological Forecasting
and Social Change, 73, 761-777. https://doi.org/10.1016/j.techfore.2005.08.003 Harrell S., Seidel T., Fay B. (1996) The National Technology Roadmap for Semiconductors and SEMATECH future directions. Microelectronic
Engineering, 30(1-4), 11-15. https://doi.org/10.1016/0167-9317(95)00185-9 Hirose Y., Phaal R., Probert D. (2015) A Conceptual Framework for Exploring the Scalable Integration of Roadmapping and Innovation System Functions for Industrial Emergence. Paper presented at the DRUID Academy Conference, Rebild, Aalborg, Denmark. https://conference. druid.dk/acc_papers/g7in5823lv9xd696hb4csdbqkjge.pdf, accessed 22.04.2021 Ho J.-Y., O'Sullivan E. (2019) Key Principles for Integrating Multiple Roadmaps for Innovation System Foresight: Case Studies of RTOs with Innovation Missions Beyond Just Technology R D. Paper presented at the 2019 Portland International Conference on Management of Engineering and Technology (PICMET). https://doi.org/10.23919/PICMET.2019.8893831 Hussain M., Tapinos E., Knight L. (2017) Scenario-driven roadmapping for technology foresight. Technological Forecasting and Social
Change, 124, 160-177. https://doi.org/10.1016/j.techfore.2017.05.005 Isenmann R. (2008) Software-Werkzeuge zur Unterstützung des Technologie-Roadmapping. In: Technologie-Roadmapping: Zukunftsstrategien für Technologieunternehmen (eds. M.G. Möhrle, R. Isenmann), Berlin, Heidelberg: Springer, pp. 229-267. https://doi. org/10.1007/978-3-540-74755-0_12 Kanama D., Kondo A., Yokoo Y. (2008) Development of technology foresight: Integration of technology roadmapping and the Delphi method. International Journal of Technology Intelligence and Planning (IJTIP), 4(2), 184-200. https://doi.org/10.1504/IJTIP.2008.018316 Kappel T.A. (2001) Perspectives on roadmaps: How organizations talk about the future. Journal of Product Innovation Management, 18,
39-50. https://doi.org/10.1111/1540-5885.1810039 Kerr C., Phaal R. (2020) Technology roadmapping: Industrial roots, forgotten history and unknown origins. Technological Forecasting and
Social Change, 155, 119967. https://doi.org/10.1016/j.techfore.2020.119967 Kostoff R.N., Schaller R.R. (2001) Science and technology roadmaps. IEEE Transactions on Engineering Management, 48, 2, 132-143. https://doi.org/10.1109/17.922473
Kunseler E.-M., Tuinstra W., Vasileiadou E., Petersen A.C. (2015) The reflective futures practitioner: Balancing salience, credibility and
legitimacy in generating foresight knowledge with stakeholders. Futures, 66, 1-12. https://doi.org/10.1016/j.futures.2014.10.006 Lee J.H., Kim H., Phaal R. (2012) An analysis of factors improving technology roadmap credibility: A communications theory assessment
of roadmapping processes. Technological Forecasting and Social Change, 79, 263-280. https://doi.org/10.1016/j.techfore.2011.05.003 Martin B.R., Irvine J. (1990) Research Foresight: Priority-Settingin Science.Prometheus,8,199-202.https://doi.org/10.1080/08109029008631897 Nimmo G. (2013) Technology Roadmapping on the Industry Level: Experiences from Canada. In: Technology Roadmapping for Strategy and Innovation (eds. M.G. Moehrle, R. Isenmann, R. Phaal), Berlin, Heidelberg: Springer, pp. 47-65. https://doi.org/10.1007/978-3-642-33923-3_4
NRC (2012) NASA Space Technology Roadmaps and Priorities: Restoring NASA's Technological Edge and Paving the Way for a New Era in
Space, Washington, D.C.: National Research Council https://doi.org/10.17226/13354 Oliveira M.G., Fleury A.L. (2015) A framework for improving the roadmapping performance. In: Proceedings of the 2015 Portland International Conference on Management of Engineering and Technology (PICMET), 2-6 Aug. 2015, Portland, OR, USA, pp. 2255-2263. https://doi.org/10.1109/PICMET.2015.7273103 Park H., Phaal R., Ho J.-Y., O'Sullivan E. (2020) Twenty years of technology and strategic roadmapping research: A school of thought
perspective. Technological Forecasting and Social Change, 154, 119965. https://doi.org/10.1016/j.techfore.2020.119965 Phaal R., Farrukh C., Probert D. (2004a) Customizing Roadmapping. Research-Technology Management, 47, 26-37. https://doi.org/10.108 0/08956308.2004.11671616
Phaal R., Farrukh C.J.P., Probert D.R. (2004b) Technology roadmapping — A planning framework for evolution and revolution. Technological
Forecasting and Social Change, 71, 5-26. https://doi.org/10.1016/S0040-1625(03)00072-6 Phaal R., Farrukh C.J.P., Probert D.R. (2007) Strategic Roadmapping: A Workshop-based Approach for Identifying and Exploring Strategic
Issues and Opportunities. Engineering Management Journal, 19(1), 3-12. https://doi.org/10.1080/10429247.2007.11431716 Phaal R., Muller G. (2009) An architectural framework for roadmapping: Towards visual strategy. Technological Forecasting and Social
Change, 76, 39-49. https://doi.org/10.1016/j.techfore.2008.03.018 Popper R. (2008) Foresight Methodology. In: The Handbook of Technology Foresight. Concepts and Practice (PRIME Series on Research and Innovation Policy) (eds. L. Georghiou, J. Cassingena Harper, M. Keenan, I. Miles, R. Popper), Cheltenham: Edward Edgar Publishing Limited, pp. 44-90.
Salmenkaita J.-P., Salo A. (2004) Emergent foresight processes: Industrial activities in wireless communications. Technological Forecasting
and Social Change, 71, 897-912. https://doi.org/10.1016/j.techfore.2003.09.001 Saritas O., Oner M.A. (2004) Systemic analysis of UK foresight results: Joint application of integrated management model and roadmapping.
Technological Forecasting and Social Change, 71, 27-65. https://doi.org/10.1016/S0040-1625(03)00067-2 Schuh G., Wemhöner H., Orilski S. (2013) Technological Overall Concepts for Future-Oriented Roadmapping, In: Technology Roadmapping for Strategy and Innovation (eds. M.G. Moehrle, R. Isenmann, R. Phaal), Berlin, Heidelberg: Springer, pp. 107-121. https://doi. org/10.1007/978-3-642-33923-3_7
Strauss J.D., Radnor M. (2004) Roadmapping for Dynamic and Uncertain Environments. Research-Technology Management, 47, 51-58.
https://doi.org/10.1080/08956308.2004.11671620 Van De Ven H. (1993) The development of an infrastructure for entrepreneurship. Journal of Business Venturing, vol. 8, (3), 211-230.
https://doi.org/10.1016/0883-9026(93)90028-4 Vishnevskiy K., Karasev O., Meissner D. (2015) Integrated roadmaps and corporate foresight as tools of innovation management: The case
of Russian companies. Technological Forecasting and Social Change, 90, 433-443. https://doi.org/10.1016/j.techfore.2014.04.011 Yasunaga Y., Watanabe M., Korenaga M. (2009) Application of technology roadmaps to governmental innovation policy for promoting technology convergence. Technological Forecasting and Social Change, 76, 61-79. https://doi.org/10.1016/j.techfore.2008.06.004
