CC BY-NC-ND
155
Social and Business Innovations: Are Common Measurement Approaches Possible?
Attila Havas
Senior Research Fellow, Centre for Economic and Regional Studies, Institute of Economics, Hungarian Academy of Sciences. Address: 1112, Budaörsi üt 45 Budapest, Hungary. E-mail: [email protected]
This article reviews various approaches to measuring business innovation with the aim of drawing lessons for measuring social innovations, and offers several methodological and policy conclusions. First, Innovation Union Scoreboard (IUS) indicators, in principle, could be useful in settings where the dominant mode of innovation is based on R&D activities. In practice, however, both R&D and non-R&D-based modes of innovation are important. IUS, therefore, only provides a partial picture. Social innovations can rely on R&D-based technological innovations; their essence, however, tends to be organizational, managerial, and behavioural changes. The IUS indicators do not capture these types of changes. Second, an assessment of the
81 indicators used to compile the Global Innovation Index reveals that it would not be fruitful to rely on such indicators to capture social innovations. Third, given the diversity among innovation systems, a poor performance signalled by a composite indicator does not automatically identify the area(s) necessitating the most urgent policy actions; only tailored, thorough comparative analyses can do so. Finally, analysts and policy makers need to be aware of the differences between measuring (i) social innovation activities (or efforts); (ii) the framework for social innovations (pre-requisites, available inputs, skills, norms, values, behavioural patterns, etc.); and (iii) the economic, societal, and environmental impacts of social innovations.
Keywords: Evolutionary economics of innovation; business innovation; social innovation; measurement of innovation; composite indicators; scoreboards; league tables; unit of analysis DOI: 10.17323/1995-459X.2016.2.58.80
Citation: Havas A. (2016) Social and Business Innovations: Are Common Measurement Approaches Possible? Foresight and STI Governance, vol. 10, no 2, pp. 58-80. DOI: 10.17323/1995-459X.2016.2.58.80
The basic objective of this article is to review established approaches to capturing and measuring various types of innovations from the angle of measuring social innovations. Social innovation is defined by the Creating Economic Space for Social Innovation (CrESSI) project as follows: "The development and delivery of new ideas (products, services, models, markets, processes) at different socio-structural levels that intentionally seek to improve human capabilities, social relations, and the processes in which these solutions are carried out." [Budd et al., 2015].
It is important to clarify at the outset that this article considers all sorts of business (or, profit-oriented) innovations on the one hand, and social (socially-oriented or societal) innovations, on the other. In other words, we should take into account not only technological (product, service, and process) innovations when discussing profit-oriented innovations, but organizational and marketing innovations as well.1 Innovation studies also show that it is more of an exception than a rule to introduce technological innovations without organizational innovations and, in many cases, marketing and market innovations as well. Moreover, the latter are vital for the success of the former [Pavitt, 1999; Tidd et al., 1997]. In particular, radical innovations often create new markets and that is, by definition, a market innovation also.
In a similar vein, technological innovations, aimed at tackling societal challenges, should not be neglected when considering social innovations. Most likely certain organizational and marketing innovations are useful — or even indispensable — to achieve societal goals. In sum, we should keep in mind a distinction between the nature of innovation (technological, organizational, and marketing) and the goals of innovation efforts (business versus societal purposes).
Significant progress has been achieved in measuring R&D and innovation activities since the 1960s with the intention to provide comparable data sets as a solid basis for assessing R&D and innovation performance and thereby guiding policy makers in devising appropriate policies [Grupp, 1998; Grupp, Schubert, 2010; Smith, 2005].2
Although there are widely used guidelines to collect data on R&D and innovation — the Frascati and Oslo Manuals [OECD, 2002, 2005] — it is not straightforward to find the most appropriate way to assess R&D and innovation performance. To start with, R&D is such a complex, multifaceted process that it cannot be sufficiently characterized by two or three indicators, and that applies to innovation a fortiori. Hence, there is always a need to select a certain set of indicators to depict innovation processes, and in particular to analyse and assess innovation performance. The choice of indicators is, therefore, an important decision reflecting the mind-set of the decision makers. These figures are 'subjective' in that respect, but as they are expressed in numbers, most people perceive indicators as being 'objective' by definition.
For this and other reasons, it is important to review how innovation is understood in particular models of innovations and analysed by various schools of economics. Based on this, the paper focuses on an in-depth assessment of two major measurement endeavours, namely the Innovation Union Scoreboard (IUS) and the Global Innovation Index (GII), and discusses further methodological issues.
Models and economic theories of innovation
Besides Schumpeter, only a few economists had perceived innovation as a relevant research topic in the first half of the 20th century.3 At that time, however, natural scientists, managers of firms' R&D laboratories, and policy advisors had formulated the first models of innovations, stressing the importance of scientific research, and these ideas are still highly influential.4 Since the late 1950s, more and more economists have shown interest in studying innovation, leading to new models of innovation, as well as making explicit mention of innovation in various economics paradigms. The role of innovation in economic
1 These three types of innovations are defined by the Oslo Manual [OECD, 2005], aimed at providing guidelines to interpret and measure innovations introduced by businesses. Interestingly, market innovations, that is, entering new markets to purchase inputs or sell outputs (not to be confused with marketing innovations) are not mentioned by the Manual (although these are parts of the classic description of innovation by Schumpeter, and important ones, indeed). Perhaps it would be almost impossible to measure these crucial innovations. Further financial innovations are not mentioned either as a separate category. Certain types of financial innovations can be interpreted as service innovations (e.g. new financial 'products'), while others (e.g. e- and m-banking) as new business practices, that is, organizational innovations using the definitions presented in the Manual.
2 "The Innovation Union Scoreboard 2013 gives a comparative assessment of the innovation performance of the EU27 Member States and the relative strengths and weaknesses of their research and innovation systems." [European Commission, 2013a, p. 4].
3 This section draws heavily on Section 2 in [Havas, 2015a].
4 For further details, see, e.g. [Fagerberg et al., 2011, p. 898] and [Godin, 2008, pp. 64-66].
development, however, is analysed by various schools of economics in diametrically different ways.5 The underlying assumptions and key notions of these paradigms lead to diverse policy implications.
Models of innovation and policy implications6
Natural scientists and practitioners had devised the first models of innovation before economists showed a serious interest in these issues. The idea that basic research is the main source of innovation had already been proposed at the beginning of the 20th century, gradually leading to what is known today as the science-push model of innovation, forcefully advocated by Bush [Bush, 1945]. It is worth recalling some of the main building blocks of Bush's reasoning:
"We will not get ahead in international trade unless we offer new and more attractive and cheaper products. Where will these new products come from? How will we find ways to make better products at lower cost? The answer is clear. There must be a stream of new scientific knowledge to turn the wheels of private and public enterprise. There must be plenty of men and women trained in science and technology for upon them depend both the creation of new knowledge and its application to practical purposes. (...) New products and new processes do not appear full-grown. They are founded on new principles and new conceptions, which in turn are painstakingly developed by research in the purest realms of science. Today, it is truer than ever that basic research is the pacemaker of technological progress. In the nineteenth century, Yankee mechanical ingenuity, building largely upon the basic discoveries of European scientists, could greatly advance the technical arts. Now the situation is different.
A nation which depends upon others for its new basic scientific knowledge will be slow in its industrial progress and weak in its competitive position in world trade, regardless of its mechanical skill." [Bush, 1945, chapter 3].
By the second half of the 1960s, the so-called market-pull model contested that reasoning, portraying demand as the driving force of innovation. Then a long-lasting and detailed discussion started to establish which of these two types of models is correct, that is, whether R&D results or market demand are the most important information sources of innovations.7
Both the science-push and the market-pull models portray innovation processes as linear ones (Figure 1). In contrast, the chain-linked model of innovation, suggested by Kline and Rosenberg [Kline, Rosenberg, 1986], stresses the non-linear properties of innovation processes, the variety of information sources, as well as the importance of various feedback loops (Figure 2).
Figure 1. Linear models of innovation
а) Science-push model
Basic science
b) Market pull model
Market need
Design and engineering
Manufacturing
Marketing Sales
Development
Manufacturing
Sales
Sources: [Dodgson, Rothwell, 1994, p. 41 (figures 4.3 and 4.4)].
5 The ensuing overview can only be brief, and thus somewhat simplified. More detailed and nuanced accounts, major achievements and synthesizing pieces of work include [Baumol, 2002; Baumol et al., 2007; Castellacci, 2008a; Dodgson, Rothwell, 1994; Dodgson et al., 2014; Dosi, 1988a, 1988b; Dosi et al., 1988; Edquist, 1997; Ergas, 1986, 1987; Fagerberg et al., 2005; Fagerberg et al., 2012; Freeman, 1994; Freeman, Soete, 1997; Grupp, 1998; Hall, Rosenberg, 2010; Klevorick et al., 1995; Laestadious et al., 2005; Lazonick, 2013; Lundvall, 1992; Lundvall, Borras, 1999; Martin, 2012; Metcalfe, 1998; Mowery, Nelson, 1999; Nelson, 1993, 1995; OECD, 1992, 1998; Pavitt, 1999; Smith, 2000; von Tunzelmann, 1995].
6 This brief account can only list the most influential models; for detailed discussions on their emergence, properties, and use for analytical and policy-making purposes see [Balconi et al., 2010; Caraga et al., 2009; Dodgson, Rothwell, 1994; Godin, 2006].
7 It is telling that a recent review of this discussion by [Di Stefano et al., 2012] draws on 100 papers.
Figure 2. The chain-linked model of innovation
D
K
C
Potential market
K
C
Invent and/or produce analytic design
Research
R p
Knowledge
K
1-2^
C
Detailed design and test
1 /2T t
Redesign and produce
I S
7
Distribute and market
F
Chain-linked model showing flow paths of information and cooperation.
Symbols on arrows: C — central-chain-of-innovation, f — feedbak loops, F — particularly important feedback.
K-R — links through knowledge to research and return paths. If problems solved at node K, link 3 to R not activated. Return from research (link 4) is problematic — therefore dashed line.
D — direct link to and from research from problems in invention and design. I — support of scientific research by instruments, machines, tools, and procedures of technology.
S — support of research in sciences underlying product ares to gain information directly and by monitoring outside work. The information obtained may apply anywhere along the chain.
Source: [Kline, Rosenberg, 1986].
4
4
The chain-link model has then been extended into the networked model of innovation. Its recent, highly sophisticated version is called the multi-channel, interactive learning model (Figure 3).
"[This model] has representational purposes and not representative ones, i.e. it does not assume that all factors have to be in place for innovation to be realised and successful. Rather, it tries to provide a stylised representation of the main classes of variables, and their interrelationships, which are involved in the innovation process taking place in a wide array of industries. (...)
Thus, the model is an analytical grid that describes and contextualises elements, but it also provides a set of flexible generalisations upon which to base our thinking when trying to explain the sources and stages of the innovation process. It points to the ubiquitous experience-based learning processes taking place within firms, as well as at the interfaces with users, suppliers and competitors." [Carafa et al., 2009, pp. 864-866; emphasis added, footnotes removed from the original].
Innovation in economics paradigms
Technological, organizational, managerial changes and the opening up of new markets had been major themes in classical economics — without using the term innovation [Havas, 2015b]. Later, neo-classical economics essentially abandoned research questions concerned with dynamics, and instead focused on static allocative efficiency. Optimization was the key issue for this school that assumed homogenous products, diminishing returns to scale, technologies accessible to all producers at zero cost, perfectly informed economic agents, perfect competition, and thus zero profit. Technological changes were treated as exogenous to the economic system, while other types of innovations were not considered at all. Given the empirical findings and theoretical work on firm behaviour and the operation of markets, mainstream industrial economics and organizational theory has relaxed the most unrealistic assumptions, especially perfect information, deterministic environments, perfect competition, and constant or diminishing returns. Yet, "this literature has not addressed institutional issues, it has a very narrow concept of
o-
IsJ
Figure. 3. The multi-channel, interactive learning model of innovation
M
O
o ©
3
►
z
n
< o
o
IsJ
IsJ O
O
Source: [Carafa et al, 2009].
uncertainty, it has no adequate theory of the creation of technological knowledge and technological interdependence amongst firms, and it has no real analysis of the role of government" [Smith, 2000, p. 75]. Evolutionary economics of innovation rests on radically different postulates compared to mainstream economics.8 The latter assumes rational agents, who can optimize by calculating risks and taking appropriate actions, while the former stresses that "innovation involves a fundamental element of uncertainty, which is not simply the lack of all the relevant information about the occurrence of known events, but more fundamentally, entails also (a) the existence of techno-economic problems whose solution procedures are unknown, and (b) the impossibility of precisely tracing consequences to actions" [Dosi, 1988a, p. 222, emphasis added]. Thus, optimization is impossible on theoretical grounds. Availability of information (symmetry vs. asymmetry among agents in this respect) has been the central issue in mainstream economics until recently. Evolutionary economics, in contrast, has stressed since its beginnings that the success of firms depends on their accumulated knowledge — both codified and tacit, skills, as well as learning capabilities. Information can be purchased (e.g. as a manual, blueprint, or licence), and hence can be accommodated in mainstream economics as a special good relatively easily and comfortably. Yet, knowledge — in particular, the types of knowledge required for innovation such as tacit knowledge, skills, and competences in pulling together and exploiting available pieces of information — cannot be bought and used instantaneously. A learning process cannot be avoided if one is to acquire knowledge and skills, and that not only takes time, but the costs of trial and error need to be incurred as well.9 Thus, the uncertain, cumulative, and path-dependent nature of innovation is reinforced. Cumulativeness, path-dependence, and learning lead to heterogeneity among firms, as well as other organizations. On top of that, sectors also differ in terms of major properties and patterns of their innovation processes [Castellacci, 2008b; Malerba, 2002; Pavitt, 1984; Peneder, 2010]. Innovators are not lonely champions of new ideas. While talented individuals may develop radically new, brilliant scientific or technological concepts, successful innovations require various types and forms of knowledge that are rarely possessed by a single organization. Close collaboration among firms, universities, public and private research organizations, and specialized service-providers is, therefore, a prerequisite for major innovations, which can take various forms - from informal communications to highly sophisticated R&D contracts, alliances, and joint ventures [Freeman, 1991, 1994, 1995; Lundvall, Borras, 1999; OECD, 2001; Smith, 2000, 2002; Tidd et al., 1997]. In other words, 'open innovation' is not a new phenomenon at all [Mowery, 2009].
Policy rationales derived from economic theories
Different policy rationales can be used from competing schools of economic thought. Mainstream economics is primarily concerned with market failures: the unpredictability of knowledge outputs from inputs, the inability to appropriate full economic benefits of private investment in knowledge creation, and the indivisibility in knowledge production lead to a 'suboptimal' level of business R&D efforts. Policy interventions, therefore, are justified if they aim at (a) creating incentives to boost private R&D expenditures by ways of subsidies and protection of intellectual property rights, or (b) funding for public R&D activities.
The evolutionary economics of innovation school investigates the role of knowledge creation and exploitation in economic processes; that is, it does not focus exclusively on R&D. This paradigm considers
8 The so-called new or endogenous growth theory is not discussed here separately because its major implicit assumptions on knowledge are very similar to those of mainstream economics [Lazonick, 2013; Smith, 2000]. Moreover, knowledge in new growth models is reduced to codified scientific knowledge, in sharp contrast to the much richer understanding of knowledge in evolutionary economics of innovation. When summarising the "evolution of science policy and innovation studies" (SPIS), Martin also considers this school as part of mainstream economics: "Endogenous growth theory is perhaps better seen not so much as a contribution to SPIS but rather as a response by mainstream economists to the challenge posed by evolutionary economics" [Martin, 2012, p. 1230].
9 Arrow [Arrow, 1962] was already discussing "The Economic Implications of Learning by Doing," and Rosenberg [Rosenberg, 1982] stressed the importance of learning by using. Recently, learning has become a more regular subject in mainstream economics, most notably in game theory. For instance, while "learning" only appeared twice in the title of NBER working papers in 1996, it occurred 5 times in 1999, 6 times in 2002, 13 times in 2008, 10 times in 2013, and 12 times in 2014. The expressions used include
"learning by doing", "learning from experience", and "learning from exporting" — but also "learning from state longitudinal data systems" and "learning millennial-style." It should be added that at least 15-20 NBER working papers are published a week. Taking the titles and abstracts of articles published in the American Economic Review, "learning" occurred first in 1999, then 2-3 times a year in 2002-2006; 4 times in 2008, 2011, and 2012; 5 times in 2013; 6 times in 2007, 2010, and 2014; and 7 times in 2009. These articles discuss a wide variety of research themes — e.g. behaviour of firms and other organizations, business cycles, stock exchange transactions, forecasting of economic growth, mortgage, art auctions, game theory, behavioural economics, energy, health, labour market — and modes of learning. Thus, not all these articles are relevant from the point of analysing innovation processes (e.g. "learning [one's] HIV status" is not part of an innovation process). Further, in several cases knowledge is narrowed down to patents, which is clearly a misconception. Yet, a detailed analysis of the substance of these articles is beyond the scope of this paper.
various types and forms of knowledge, including practical or experience-based knowledge acquired via learning by doing, using, and interacting. As these are all relevant to innovation, scientific knowledge is far from being the only type of knowledge required for successfully introducing new products, processes or services, let alone non-technological innovations. R&D is undoubtedly among the vital sources of knowledge. Besides in-house R&D projects, however, results of other R&D projects are also widely utilized during the innovation process: extramural projects conducted in the same or other sectors, at public or private research establishments, home or abroad. More importantly, there are a number of other sources of knowledge that are also essential for innovations, including design, scaling up, testing, tooling-up, trouble-shooting, and other engineering activities, ideas from suppliers and users, inventors' concepts and practical experiments [Hirsch-Kreinsen et al., 2005; Klevorick et al., 1995; Lundvall, 1992; Lundvall, Borras, 1999; Rosenberg, 1986, 1998; von Hippel, 1988]. Collaborations among engineers, designers, artists, and other creative 'geeks' are also significant sources of knowledge. Further, innovative firms also utilize knowledge embodied in advanced materials and other inputs, equipment, and software. The Community Innovation Survey (CIS) defines its own set of categories as highly important sources of information for product and process innovation. These include the enterprise or the enterprise group; suppliers of equipment, materials, components or software; clients or customers; competitors or other enterprises from the same sector; consultants, commercial labs or private R&D institutes; universities or other higher education institutes; public research institutes; conferences, trade fairs, exhibitions; scientific journals and trade/ technical publications; and professional and industry associations. All rounds of CIS clearly and consistently show that firms regard a wide variety of information sources as highly important ones for innovation. This paper, however, given space limits, only presents the 2010-2012 data in Figures 4-5.10
The wide variety of knowledge drawn on in innovation processes is a crucial point to bear in mind as the OECD classification of industries only takes into account expenditures on formal R&D activities, carried out within the boundaries of a given sector.11 In other words, a number of highly successful, innovative firms, exploiting advanced knowledge created externally in distributed knowledge bases [Smith, 2002] and internally by non-R&D processes, are classified as medium-low-tech or low-tech, because their R&D expenditures are below the threshold set by the OECD.
In sum, evolutionary economics of innovation posits that the success of firms is largely determined by their abilities to exploit various types of knowledge, generated by both R&D and non-R&D activities. Knowledge generation and exploitation takes place in, and is fostered by, various forms of internal and external interactions. The quality and frequency of the latter is largely determined by the properties of a given innovation system, in which these interactions take place. STI policies, therefore, should aim at strengthening the respective innovation system and improving its performance by tackling systemic
Figure 4. Highly important 'business' sources of information for product and process innovation, selected EU members, 2010-2012 (share of respondents selecting each response out
of total respondents by country, %)
10 Data for the 2006-2008 and 2008-2010 periods are presented in [Havas, 2015c].
11 The so-called indirect R&D intensity has been also calculated as R&D expenditures embodied in intermediates and capital goods purchased on the domestic market or imported. Yet, it has been concluded that indirect R&D intensities would not influence the classification of sectors [Hatzichronoglou, 1997, p. 5].
Figure 5. Highly important 'scientific' sources of information for product and process innovation, selected EU members, 2010-2012 (share of respondents selecting each response out of
total respondents by country, %)
failures hampering the generation, diffusion, and utilisation of any type of knowledge required for successful innovation.12 To put it in another way, conscious, co-ordinated policy efforts are needed to promote knowledge-intensive activities in all sectors.
The Innovation Union Scoreboard
As shown above, firms exploit various types of knowledge for their innovation activities. Applying this general observation to the Danish case, and relying on the DISKO survey, Jensen et al. [Jensen et al., 2007] made an elementary distinction between two modes of innovation: (a) one based on the production and use of codified scientific and technical knowledge (in brief, the ST[I] mode), and (b) another one relying on informal processes of learning and experience-based know-how (called DUI: learning by Doing, Using and Interacting).
Following this distinction, the indicators used in the various editions of the Innovation Union Scoreboard13 are characterized below, using a rudimentary classification. An indicator can be relevant to reflect:
• only R&D-based innovations
• mainly R&D-based innovations
• only non-R&D-based innovations
• mainly non-R&D-based innovations
• both types of innovations
This rudimentary classification reveals a bias towards R&D-based innovations in the first edition of the EIS: 10 indicators were only relevant for R&D-based innovations; 8 could be relevant for both types of innovations; and none focused on non-R&D-based innovations (Table 1).14
The 2014 and 2015 editions of the IUS use 25 indicators, grouped by 8 innovation dimensions [European Commission, 2014, 2015]. Repeating the same exercise shows that the bias towards R&D-based innovations has been retained: ten of the most recent IUS indicators15 are only relevant for, and a further
12 In an attempt to systematically compare the market and systemic failure policy rationales, Bleda and del Río [Bleda, del Río, 2013] introduce the notion of evolutionary market failures, and reinterpret "the neoclassic market failures" as particular cases of evolutionary market failures, relying on the crucial distinction between knowledge and information.
13 The Innovation Union Scoreboard was originally called the European Innovation Scoreboard (EIS). The EIS and IUS indicators have been revised several times since the first edition of the scoreboard of EIS 2002 [European Commission, 2002]. The current name of the scoreboard was introduced in 2010.
14 A fairly detailed, partly technical, partly substantive discussion would be needed to refine this simple classification.
15 There was only a slight change introduced in 2015: the indicator called "Contribution of medium and high-tech product exports to the trade balance" was replaced by "Exports of medium and high-technology products as a share of total product exports." This change had no effect on the nature of the indicators, and thus the 2014 edition of the IUS is not reproduced here.
four mainly capture, R&D-based innovations; seven could be relevant for both types of innovations; and a mere four focus on non-R&D-based innovations (Table 2).
A fairly detailed discussion, both technical and substantive, would be needed to refine this simple classification, particularly concerning the following issues:
• To what extent upper secondary education, venture capital, employment in knowledge-intensive activities, and knowledge-intensive services exports are relevant indicators to capture non-R&D-based innovations?
• How much are non-R&D-based innovation activities needed for successful R&D-based innovations? To give an overview of the evolution of the EIS and IUS indicators, results are summarized in Table 3.16 Overall, the bias towards R&D-based innovations has been rather persistent, although there has been some fluctuation.
Тable 1. The 2002 European Innovation Scoreboard indicators
Relevance for R&D- based innovation Relevance for non-R&D- based innovation
1 Human resources
New S&E graduates (ISCED 5a and above) per 1000 population aged 20-29 X
Population with tertiary education (% of 25-64 years age class) b b
Participation in life-long learning (% of 25-64 years age class) b b
Employment in medium-high and high-tech manufacturing (% of total workforce) X
Employment in high-tech services (% of total workforce) X
2 Knowledge creation
Public R&D expenditures (GERD - BERD) (% of GDP) X
Business expenditures on R&D (BERD) (% of GDP) X
EPO high-tech patent applications (per million population) X
USPTO high-tech patent applications (per million population) X
3 Transmission and application of knowledge
SMEs innovating in-house (% of manufacturing SMEs) b b
SMEs involved in innovation co-operation (% of manufacturing SMEs) b b
Innovation expenditures (% of all turnover in manufacturing) b b
4 Innovation finance, output and markets
High technology venture capital investment (% of GDP) X
Capital raised on parallel markets plus by new firms on main markets (% of GDP)* X
Sales of 'new to market' products (% of all turnover in manufacturing) b b
Home internet access (% of all households) b b
ICT expenditures (% of GDP) b b
Share of manufacturing value-added in high-tech X
Legend: X: only relevant x: mainly relevant b: relevant for both types Notes: Public R&D expenditures do not equal to GERD - BERD; rather, it should be the sum of government-funded parts of BERD, GOVERD, and HERD. Three indicators, namely EPO patent applications (per million population), Home internet access (per 100 population), and Inward FDI stock (% of GDP) were only used for candidate countries. * "Parallel stock exchanges focus on high technology sectors." [European Commission, 2002, p. 31]. Source: own compilation, drawing on the detailed definition of indicators [European Commission, 2002].
16 The indicators used in the previous editions of the EIS and IUS are characterized in detail in [Havas, 2015c].
Тable 2. The 2015 Innovation Union Scoreboard indicators
Relevance for R&D-based innovation Relevance for non-R&D-based innovation
Human resources
New doctorate graduates (ISCED 6) per 1000 population aged 25-34 X
Percentage population aged 30-34 having completed tertiary education b b
Percentage youth aged 20-24 having attained at least upper secondary level education b b
Open, excellent and attractive research systems
International scientific co-publications per million population X
Scientific publications among the top 10% most cited publications worldwide as % of total scientific publications of the country X
Non-EU doctorate students* as a % of all doctorate students X
Finance and support
R&D expenditure in the public sector as % of GDP X
Venture capital investment as % of GDP x
Firm investments
R&D expenditure in the business sector as % of GDP X
Non-R&D innovation expenditures as % of turnover X
Linkages & entrepreneurship
SMEs innovating in-house as % of SMEs b b
Innovative SMEs collaborating with others as % of SMEs b b
Public-private co-publications per million population X
Intellectual assets
PCT patents applications per billion GDP (in PPS€) X
PCT patent applications in societal challenges per billion GDP (in PPS€) (environment-related technologies; health) X
Community trademarks per billion GDP (in PPS€) X
Community designs per billion GDP (in PPS€) X
Innovators
SMEs introducing product or process innovations as % of SMEs b b
SMEs introducing marketing or organizational innovations as % of SMEs X
Economic effects
Employment in fast-growing enterprises in innovative sectors (% of total employment) b b
Employment in knowledge-intensive activities (manufacturing and services) as % of total employment x
Exports of medium and high-technology products as a share of total product exports x
Knowledge-intensive services exports as % total service exports x
Sales of new to market and new to firm innovations as % of turnover b b
License and patent revenues from abroad as % of GDP X
Legend: X: only relevant x: mainly relevant b: relevant for both types * It is a somewhat strict definition of openness, which only takes into account non-EU doctorate students. Source: own compilation.
Тable 3. The evolution of the EIS and IUS indicators, 2002-2014
EIS 2002 EIS 2003 EIS 2004 EIS 20052006 EIS 2007 EIS 2008 EIS 2009 IUS 20102013 IUS 20142015
Indicators reflecting:
only R&D-based innovations 10 9 9 8 7 8 8 10 10
mainly R&D-based innovations - 3 3 5 5 4 4 4 4
both types 8 9 9 12 12 15 16 6 7
only non-R&D-based innovations - - - - - 1 1 4 4
mainly non-R&D-based innovations - - 1 1 1 1 1 - -
Number of indicators 18 21 22 26 25 29 30 24 25
Source: own compilation.
Several conclusions can be drawn from the above considerations for analysing social innovation. First, while the number and definitions of indicators used to compile the various editions of EIS and IUS have changed considerably since 2002, these indicators consistently focus on measuring R&D activities (inputs and outputs) and R&D-based innovation activities. In other words, they can be relevant in settings characterized predominantly by the so-called ST mode of innovation, but are significantly less useful in other settings that are characterized by other types of innovation activities. In other words, using the EIS or IUS indicators would not help find out whether a certain system is characterized by a low level of innovation activities altogether or a low level of R&D-based innovation activities. Yet, that is an important distinction from an analytical as well as a practical (policy) point of view: these two systems (settings) are fundamentally different.
Several analysts and policy makers tend to believe that (a) advanced economies can be sufficiently characterized by focusing on the ST mode of innovation, and (b) less advanced economies should attempt to change the sectoral composition of their economy by increasing the weight of the so-called high-tech (HT) sectors. These views, however, cannot be corroborated by empirical evidence. Any simple statistical analysis reveals that the so-called high-tech sectors that are supposed to be drivers of economic development, due their intense ST mode of innovation activities, have a fairly low weight either in output or employment. Innovation studies have shown that technological innovations can hardly be introduced without organizational and managerial innovations. Moreover, the latter ones — together with marketing innovations — are vital for the success of the former17 [Pavitt, 1999; Tidd et al., 1997]. In fact, those companies that consciously combine the ST and DUI modes of innovation are the most successful ones [Jensen et al., 2007].
Yet, the high-tech myth is so powerful that even researchers who base their work on thorough analyses of facts are surprised when the facts are at odds with the widespread obsession with high-tech. A telling example is Peneder's excellent study on the 'Austrian paradox':
"On the one hand, macroeconomic indicators on productivity, growth, employment and foreign direct investment indicate that overall performance is stable and highly competitive. On the other hand, an international comparison of industrial structures reveals a severe gap in the most technologically advanced branches of manufacturing, suggesting that Austria is having problems establishing a foothold in the dynamic markets of the future" [Peneder, 1999, p. 239]. In contrast, evolutionary economics of innovation claims that any firm — belonging to either a low-and medium-technology (LMT) or a HT sector — can become competitive in 'the dynamic markets of the future' if it successfully combines its own firm-specific innovative capabilities with 'extra-mural' knowledge available in distributed knowledge bases. In other words, Austrian policy makers need not be concerned with the observed 'paradox' as long as they help firms sustain their learning capabilities, and thereby maintain their innovativeness. That would lead to good economic performance irrespective of the share of LMT industries in the Austrian economy.
17 Although it goes without saying that not all technological innovations are based on R&D results, people tend to forget this basic fact. Certain organizational, managerial, marketing, and financial innovations, in turn, draw on R&D results (but usually not stemming from R&D activities conducted or financed by firms). For these two reasons, it would be a mistake to equate technological innovations with R&D-based innovations.
From a different angle, while the bulk of innovation activities in the LMT sectors are not based on intramural R&D efforts, these sectors also improve their performance by various types of innovations. These firms are usually engaged in the DUI mode of innovation, but also draw on advanced S&T results available through the so-called distributed knowledge bases [Robertson, Smith, 2008; Smith, 2002], as well as advanced materials, production equipment, software and various other inputs (e.g. electronics components and sub-systems) supplied by HT industries [Bender et al., 2005; Hirsch-Kreinsen et al., 2005; Hirsch-Kreinsen, Jacobson, 2008; Hirsch-Kreinsen, Schwinge, 2014; Jensen et al., 2007; Kaloudis et al., 2005; Mendonga, 2009; Sandven et al., 2005; von Tunzelmann, Acha, 2005]. Thus, demand by the LMT sectors constitutes major market opportunities for HT firms, and provides strong incentives and ideas for their RTDI activities [Robertson et al., 2009].
It is worth recalling that the 2003 EIS report also stressed the importance of the LMT sectors, as well as the significance of their innovation activities:
„The EIS has been designed with a strong focus on innovation in high-tech sectors. Although these sectors are very important engines of technological innovation, they are only a relatively small part of the economy as measured in their contribution to GDP and total employment. The larger share of low and mediumtech sectors in the economy and the fact that these sectors are important users of new technologies merits a closer look at their innovation performance. This could help national policy makers with focusing their innovation strategies on existing strength and overcome areas of weakness" [European Commission, 2003a, p. 20].
Since then, however, these ideas have been given less prominence. It would be an interesting research question to investigate this development, but that is beyond the scope of this paper. More recently, another European Commission (EC) document, namely the 2013 EU Competitiveness Report, gives 'mixed' messages on these issues. At certain points, it reinforces these adverse effects:
„The EU has comparative advantages in most manufacturing sectors (15 out of 23) accounting for about three quarters of EU manufacturing output. (...) Of the 15 sectors with comparative advantages mentioned above, about two-thirds are in the low-tech and medium-low tech manufacturing groups. On a positive note though, even in those sectors EU competitiveness is based on high-end innovative products" [EC, 2013b, pp. 3-4, emphasis added].
Is it a negative phenomenon, then, that around 10 EU LMT sectors are internationally competitive? A more balanced view is also offered:
"The policy priority attached to key enabling technologies which lead to new materials and products in all manufacturing sectors has a strong potential to upgrade EU competitiveness not only in the high-tech sectors but also in the traditional industries" [Ibid., p. 5]. To sum up the first conclusion, analysts and policy makers dealing with innovation should pay attention to both R&D-based and non-R&D-based innovations.
Second, while social innovations can indeed utilize R&D-based technological innovations, their essence tends to be organizational, managerial, and behavioural changes. The EIS and IUS indicators, in turn, do not capture these types of changes.
The Global Innovation Index
Compared to the IUS, the Global Innovation Index (GII) has a significantly broader coverage— in two respects: it covers well over 100 countries, and considers 81 indicators, arranged in 7 "pillars" each with 3 sub-pillars (Table 4, Figure 6).
To assess the relevance of these 81 indicators, and especially the 'match' between the themes (or headings) captured by the 7 pillars, is beyond the scope of this paper. In other words, the GII indicators are characterized in a somewhat simplified way here.18 It should be stressed that most elements are indices themselves i.e. not 'stand-alone' indicators. In other words, several methodological weaknesses are likely to remain hidden.
18 For some more detailed comments, see Appendix 3 in [Havas, 2015c].
Тable 4. Distribution of GII 2014 indicators by pillars
Name of the pillar Number of indicators
Institutions 9
Human Capital and Research 11
Infrastructure 10
Market Sophistication 10
Business Sophistication 14
Knowledge and Technology Outputs 14
Creative Outputs 13
Source: own compilation.
Pillar 1: Institutions
The first sub-pillar — political environment — incorporates three indices with the intention to reflect the following aspects: "perceptions of the likelihood that a government might be destabilized; the quality of public and civil services, policy formulation, and implementation; and perceptions of violations to press freedom".
The second sub-pillar, called regulatory environment, is comprised of two indices to capture "perceptions on the ability of the government to formulate and implement cohesive policies that promote the development of the private sector and at evaluating the extent to which the rule of law prevails (in aspects such as contract enforcement, property rights, the police, and the courts)." A third indicator is meant to evaluate "the cost of redundancy dismissal as the sum, in salary weeks, of the cost of advance notice requirements added to severance payments due when terminating a redundant worker." The third sub-pillar — business environment — is aimed at summarizing three aspects directly affecting private entrepreneurial endeavours. It uses the World Bank indices "on the ease of starting a business; the ease of resolving insolvency (based on the recovery rate recorded as the cents on the dollar recouped by creditors through reorganization, liquidation, or debt enforcement/foreclosure proceedings); and the ease of paying taxes" [Cornell University et al., 2014, pp. 45-46].
Not all the above elements are institutions ("rules of the game"), and not all are directly related to innovation processes and performance. However, it can be argued that the aspects captured by these indices may help characterize the political, regulatory, and business environment for innovation. The important missing elements include legislation on competition19 as well as the entrepreneurial culture in a given country.
Pillar 2: Human capital and research
Sub-pillar 2.1 is composed of several of indicators with the intention to capture achievements at the first two levels of education, namely elementary and secondary education. Education expenditure and school life expectancy are taken as "good proxies for coverage." Government expenditure per pupil at secondary level is meant to indicate "the level of priority given to secondary education by the state." The quality of education is measured via (a) PISA (OECD Programme for International Student Assessment) results indicating 15-year-old students' performances in reading, mathematics, science, and computing, as well as (b) the pupil-teacher ratio.
Sub-pillar 2.2 on tertiary education is designed to measure coverage at this level of education. "[P]riority is given to the sectors traditionally associated with innovation (with a series on the percentage of tertiary graduates in science and engineering, manufacturing, and construction); and the inbound mobility of tertiary students, which plays a crucial role in the exchange of ideas and skills necessary for innovation." Sub-pillar 2.3 on R&D is meant to indicate the level and quality of R&D activities by using the number of researchers (headcounts, per million of population), gross expenditures on R&D as percentage of GDP, and the quality of scientific and research organizations proxied by the average score of the top three universities in the QS World University Ranking as of 2013. "[T]his indicator aims at capturing the availability of at least three higher education institutions of quality within each economy (i.e., included in the global top 700), and is not aimed at assessing the average level of all institutions within a particular economy" [Cornell University et al., 2014, pp. 46-47].
19 The intensity of competition is included in Pillar 4.
Source: [Cornell University et al., 2014].
Formal education is undoubtedly a crucial factor determining the quality of human capital but lifelong learning and other informal modes of learning are also important. Research is conducted outside universities, too, both by publicly financed research organizations and businesses. Moreover, the quality of research conducted by these latter types of organizations is not necessarily lower than that at universities. Moreover, university rankings themselves suffer from several major methodological weaknesses. Thus, the name of this pillar is more 'ambitious' than its actual content.
Pillar 3: Infrastructure
Three sub-pillars form the third pillar of infrastructure: information and communication technologies (ICT), general infrastructure, and ecological sustainability. Sub-pillar 3.1 on ICT is computed by using four indices developed by international organizations on ICT access, ICT use, online service by governments and participation of citizens. Sub-pillar 3.2 on general infrastructure is composed of "the average of electricity output in kWh per capita; a composite indicator on logistics performance; and gross capital formation, which consists of outlays on additions to the fixed assets and net inventories of the economy, including land improvements (fences, ditches, drains); plant, machinery, and equipment purchases; and the construction of roads, railways, and the like, including schools, offices, hospitals, private residential dwellings, and commercial and industrial buildings." Sub-pillar 3.3 on ecological sustainability is constructed by using three indicators: "GDP per unit of energy use (a measure of efficiency in the use of energy), the Environmental Performance Index of Yale and Columbia Universities, and the number of certificates of conformity with standard ISO 14001 on environmental management systems issued" [Cornell University et al., 2014, p. 47].
Ecological sustainability is certainly an important issue, but it is difficult to grasp why it is part of the "infrastructure" pillar, especially when it is measured by the above three components. These are more pertinent to environmental challenges that require innovation efforts or to the outcome of previous eco-innovation efforts. In other words, there is a certain mismatch between the name of this pillar and its actual content.
Pillar 4: Market sophistication
The fourth pillar integrates three sub-pillars "structured around market conditions and the total level of transactions." Sub-pillar 4.1 on credit intends to reflect, "the ease of getting credit aimed at measuring the degree to which collateral and bankruptcy laws facilitate lending by protecting the rights of borrowers and lenders, as well as the rules and practices affecting the coverage, scope, and accessibility of credit information." Transactions are measured by the total value of domestic credit to the private sector (as a percentage of GDP) as well as by the gross loan portfolio of microfinance institutions (as a percentage of GDP) with the intention to make the method applicable to emerging markets, too. Sub-pillar 4.2 on investment is composed of the ease of protecting investors index and three indicators on the level of transactions. Besides stock market capitalization, the total value of shares traded (as percentage of GDP) is also taken into account to show if market size is matched by market dynamism. Data on venture capital deals (a total of 18,860 deals in 71 countries in 2013) are also used. Sub-pillar 4.3 considers trade and competition. The market conditions for trade are measured by two indicators: the average tariff rate weighted by import shares, and a metric on the conditions of non-agricultural foreign market access (five major export markets weighted actual applied tariffs for non-agricultural exports). The last indicator, that is, the intensity of competition in local markets, is taken from a survey as: "Efforts made at finding hard data on competition have so far proved unsuccessful" [Cornell University et al., 2014, p. 48].
Pillar 5: Business sophistication
The fifth pillar is intended to capture the level of business sophistication to assess "how conducive firms are to innovation activity". Sub-pillar 5.1 on knowledge workers is composed of four indicators: employment in knowledge-intensive services; the availability of formal training at the firm level; R&D performed by business enterprise (BERD) as a percentage of GDP; and the percentage of gross expenditures of R&D (GERD) financed by businesses. Further, it includes an indicator taken from the Graduate Management Admission Test (GMAT): "The total number of GMAT test takers (scaled by population aged 20 to 34 years old) [was] taken as a proxy for the entrepreneurial mind-set of young graduates." Sub-pillar 5.2 on innovation linkages exploits data on business-university R&D collaborations, "the prevalence of well-developed and deep clusters," the ratio of GERD financed from abroad, and "the number of deals on joint ventures and strategic alliances. The latter covers 2,978 deals announced in 2013, with firms headquartered in 127 participating economies. In addition, the total number of Patent Cooperation Treaty (PCT) and national office published patent family applications filed by residents in at least three offices is included this year to proxy for international linkages."
"The rationale behind sub-pillars 5.3 on knowledge absorption (an enabler) and 6.3 on knowledge diffusion (a result) — two sub-pillars designed to be mirror images of each other — is precisely that together they will reveal how good countries are at absorbing and diffusing knowledge. Sub-pillar 5.3 includes four statistical indicators that are linked to sectors with high-tech content or are key to innovation. These include royalty and license fees payments as a percentage of total trade; high-tech imports (net of reimports) as a percentage of total imports; imports of communication, computer and information services as a percentage of total trade; and net inflows of foreign direct investment (FDI) as a percentage of GDP" [Cornell University et al., 2014, pp. 48-49; some obvious mistakes, e.g. mentioning GERD instead of BERD, have been corrected by the author].
The name of this pillar is not explained, although it does not seem to be self-explanatory. It is not clear either, why firms should be conducive to innovation activity. Usually analyses have a different logic: market and regulatory conditions, which are external to the firms, can be conducive for - or alternatively hamper - innovation activities performed by businesses. Further, it is difficult to accept the ratio of GMAT test takers "as a proxy for the entrepreneurial mind-set of young graduates." The name of sub-pillar 5.2 (innovation linkages) only partially matches its components, of which two concern R&D activities, and a third (on patents) is more relevant to R&D than innovation activities. Data on high-tech imports can only partially reflect knowledge absorption.
Pillar 6: Knowledge and technology outputs
Sub-pillar 6.1 on knowledge creation "includes five indicators that are the result of inventive and innovative activities: patent applications filed by residents both at the national patent office and at the international level through PCT; utility model applications filed by residents at the national office; scientific and technical published articles in peer-reviewed journals; and an economy's number of articles (H) that have received at least H citations."
Sub-pillar 6.2 on knowledge impact is meant to measure "the impact of innovation activities at the micro-and macro-economic level or related proxies: increases in labour productivity, the entry density of new firms, spending on computer software, and the number of certificates of conformity with standard ISO 9001 on quality management systems issued." The indicator on the share of high- and medium-high-tech industrial output in total manufacturing output features for the first time in this edition of the GII. Sub-pillar 6.3 on knowledge diffusion "is the mirror image of the knowledge absorption sub-pillar of Pillar 5. It includes four statistics all linked to sectors with high-tech content or those that are key to innovation: royalty and license fees receipts as a percentage of total trade; high-tech exports (net of reexports) as a percentage of total exports (net of re-exports); exports of communication, computer and information services as a percentage of total trade; and net outflows of FDI as a percentage of GDP" [Cornell University et al., 2014, pp. 49-50].
The first sub-pillar comprises indicators on "the result of inventive and innovative activities." Yet, most of these indicators are relevant to characterize R&D (and not innovation) activities. As for the knowledge impact sub-pillar, only one of the five components is related to knowledge impacts, and even that one is only partially: reflecting the impact of certain types of knowledge. As for the knowledge diffusion sub-pillar, all its four components can indicate knowledge diffusion outside a given country (with certain limitations), and thus none seems to be a relevant indicator of knowledge diffusion inside a given country.
Pillar 7: Creative outputs
The first sub-pillar on intangible assets includes data on trademark applications by residents at the national office; trademark applications under the Madrid System by country of origin; and results obtained via two survey questions on the use of ICTs by businesses.
Sub-pillar 7.2 on creative goods and services aims to capture creativity and an economy's creative outputs by using five indicators: i) cultural and creative services exports, including information services, advertising, market research and public opinion polling, and other personal, cultural, and recreational services, as a percentage of total trade; ii) national feature films produced in a given country per capita; iii) global entertainment and media output per capita; iv) printing and publishing output, as a percentage of total manufacturing output; and v) creative goods exports, as a percentage of total trade. Sub-pillar 7.3 on online creativity is composed of four indicators among population aged 15-69 years: generic (biz, info, org, net, and com) and country-code top level domains, average monthly edits to Wikipedia, and video uploads on YouTube. "Attempts made to strengthen this sub-pillar with indicators in areas such as blog posting, online gaming, the development of applications, and have so far proved unsuccessful" [Cornell University et al., 2014, pp. 50-51].
It is not clear why "the use of ICTs in business and organizational models" is an output indicator. Only a small fraction of printing and publishing output is a creative output, with the bulk being revenues to cover printing costs (paper, other raw materials, and labour). It would be rather costly to establish what portion of video uploads on YouTube constitutes creative output.
In sum, the GII is a remarkable effort both in terms of its geographic and thematic coverage, but it suffers from severe weaknesses concerning business innovation activities. In several cases, there is a non-negligible mismatch between the 'headline' notions (pillars and their sub-pillars) and the actual components (indices or indicators) selected. Just as in the case of the EIS and IUS indicators, there is a bias towards R&D-based (ST mode) innovations, and thus the DUI mode is eclipsed. It is even worse when R&D and innovation are conflated. When it comes to describing and assessing social innovations, none of the 81 GII indicators are really fit for purpose.
Further methodological considerations Degree of novelty, unit of analysis
A standard question in innovation surveys relates to the degree of novelty. A given innovation can be new to the firm, to the market (in a given country) or to the world. For pragmatic reasons, the Community Innovation Survey (CIS) uses only the first two categories: it would be too difficult to judge by the respondents — and subsequently get it checked by experts — if a given innovation is new to the market in a given country or the world. Of course, in rare instances such as the introduction of the first digital camera, mobile phone, or tablet, it would be easier to establish novelty; however, even in these exceptional cases there could be some difficulties to establish which product variation (by which company) has been introduced first — and which successfully.
This issue is related closely to the classification of innovations. In qualitative analyses the following categories can be used. New goods (that is, products or services) might represent an incremental or a radical change (innovation). If we consider further units (levels) of analysis, we can also think of innovations at the level of technology systems. This would include a set of technologically and economically interconnected goods and processes affecting several companies or an entire sector at the same time, occasionally leading to the emergence of new industries (e.g. canals, gas and electric light systems, plastic goods, electric household devices). Dissatisfied with the notion of 'long waves' used in analysing business cycles (mainly by Kondratiev and Schumpeter), Freeman and Perez elaborated on the notion of techno-economic paradigms:
"The set of the most successful and profitable practices in terms of choice of inputs, methods and technologies, and in terms of organisational structures, business models and strategies. These mutually compatible practices, which turn into implicit principles and criteria for decision-making, develop in the process of using the new technologies, overcoming obstacles and finding more adequate procedures, routines and structures. The emerging heuristic routines and approaches are gradually internalised by engineers and managers, investors and bankers, sales and advertising people, entrepreneurs and consumers. In time, a shared logic is established; a new 'common sense' is accepted for investment decisions as well as for consumer choice. The old ideas are unlearned and the new ones become 'normal'" [Perez, 2010, p. 194]. As an illustration, the examples of such paradigmatic changes are the (first) industrial revolution; the age of steam and railways; the age of steel, electricity, and heavy engineering; the age of oil, automobile, and mass production; and more recently, the age of info-communications.
Some of these considerations might be useful when analysing social innovations in a qualitative way. Yet, compared to technological innovations, it is likely to be even more difficult to establish the degree of novelty of a given social innovation. The degree of novelty, however, seems to be of lesser importance in these cases: usually intellectual property rights are not an issue for social innovators. Of course, prestige — being inventive and obtaining acknowledgments for that — might play a role: it could give some impetus to be involved in certain social innovation projects. It is an empirical question to establish the role of prestige in these endeavours.
What seems to be perhaps more relevant but probably even more difficult than in the case of technological innovations is identifying whether a given social innovation is an 'isolated' new solution or a part of a new 'social system.' In other words, whether it is part of a set of socially, institutionally, organizationally, and economically interconnected social innovations that affect several groups of people or an entire community (a neighbourhood, village, town or city) at the same time. Occasionally, a new 'social system' can also lead to the emergence of new social structures, norms, institutions, behaviour, value systems and practices at a higher level of aggregation (e.g. sub-national regions, nations or even supra-national regions, for example, the European Union).
Some aspects of the notion of techno-economic paradigms are contested among economists and economic historians dealing with technological innovations, on the one hand, and this notion is probably too complex, too demanding - too far-fetched - to be applied to analyse social innovations, on the other. However, one of its features could be considered as a useful guiding principle when analysing social innovations, namely the interconnectedness of technological, organizational, and business model innovations, together with the emergence of a new, widely accepted 'common sense.' Most of the indicators and indices used to compile the Summary Innovation Index (EIS, IUS), the Global Innovation Index, and the Technology Achievement Index [UNDP, 2001] reflect the macro level. These components are calculated by aggregating micro level data (e.g. economic indicators at the firm level, and education indicators at the level of individuals). In contrast, social innovations are usually monitored (observed) at a project level. It is hardly possible to aggregate these data (observations) in a meaningful way to arrive at the macro level.
Innovation activities, their framework conditions and impacts
Despite the relatively long-established tradition in measuring business innovations and the significant efforts devoted to advance and standardize methods, there is a considerable lack of clarity on whether a certain measurement or monitoring exercise (a set of indicators, data collection, measurement and analytical methods) aims to characterize: (a) innovation activities (efforts) themselves; (b) the framework conditions (e.g. pre-requisites, available inputs, skills) of being innovative (or successful in innovation efforts); or (c) the economic, societal, or environmental impacts of innovations. Given the complexity of innovation processes themselves, as well as that of economic, societal or environmental developments, it is certainly a major difficulty to attribute a certain economic, societal or environmental phenomenon as a direct (or major) effect of a given innovation project (or a set of them at an aggregated level).
These fundamental methodological difficulties certainly apply to social innovations, too, perhaps even a fortiori. Again, a noteworthy issue is the lack of conscious efforts to distinguish between measuring (a) social innovation activities (efforts) themselves; (b) the framework conditions (e.g. pre-requisites, available inputs, skills, norms, values, behavioural patterns) of being socially innovative; and (c) the economic, societal or environmental impacts of social innovations.
Composite indicators
There is a strong, often explicit, pressure to devise so-called composite indicators to compress information into a single figure in order to compile eye-catching, easy-to-digest scoreboards. A major source of complication is choosing an appropriate weight for assigning to each component. By conducting sensitivity analyses of the 2005 European Innovation Scoreboard (EIS) [European Commission, 2005], Grupp and Schubert [Grupp, Schubert, 2010, p. 72] have shown how changing weights de-stabilizes the rank configuration. Besides assigning weights, three other ranking methods are also widely used, namely: unweighted averages, Benefit of the Doubt (BoD), and principal component analysis. Comparing these ranking methods, the authors conclude: "Not only utilizing the rankings highly sensitive to weighting (...), but even using accepted approaches like BoD or factor analysis may result in drastically changing rankings" [Ibid., p. 74]. Hence, they propose using multidimensional representations, e.g. spider charts to reflect the multidimensional character of innovation processes and performance. That would enable analysts and policy makers to identify strengths and weaknesses, and thus pinpoint more precise targets for policy actions.
Other researchers also emphasize the need for sufficiently detailed characterizations of innovation processes. For example, a family of five indicators — R&D, design, technological, skill, and innovation intensities — offers a more diversified picture on innovativeness than the Summary Innovation Index of the EIS [Laestadius et al., 2005]. Using Norwegian data, they demonstrate that the suggested method can capture variety in knowledge formation and innovativeness both within and between sectors. Thus, it supports a more accurate understanding of creativity and innovativeness and directs policy makers' attention across various sectors, and directs policy makers' attention to this diversity (suppressed by the OECD classification of sectors).This suggested method, therefore, can better serve policy design. These considerations could also be applicable to social innovations.
Conclusions
This paper has reviewed business innovation indicators from theoretical and policy perspectives. It has discussed two widely used sets of innovation indicators, their context and shortcomings, and considered whether they can be followed as a 'model' when designing social innovation indicators. The main findings can be summarized as follows. Various economics paradigms treat (business) innovation in diametrically dissimilar ways: they consider different notions as crucial ones (e.g. risk vs. uncertainty, information vs. various forms, types and sources of knowledge, skills and learning capabilities and processes); offer diverse justifications (policy rationales) for state interventions; and interpret the significance of various types of inputs, efforts, and results differently. Thus, they implicitly identify different 'targets' for measurement, monitoring, and analytical purposes (what phenomena, inputs, capacities, processes, outcomes and impacts are to be measured and assessed).
The science-push model of innovation, reinforced by the sophisticated — and thus appealing and compelling — models of mainstream economics emphasizes the economic impacts of R&D-based innovation efforts, and advances the market failure argument and the concomitant set of policy advice. Hence, it focuses the attention of decision makers and analysts on the so-called ST mode of innovation. Measurement and monitoring systems influenced by this way of thinking — most notably the Innovation Union Scoreboard of the European Commission, the Global Innovation Index and the Technology Achievement Index [UNDP, 2011] - tend to pay attention mainly to the ST mode of innovation at the expense of the so-called DUI mode of innovation. It is a major concern, however, as the latter also plays an important role in enhancing productivity, creating jobs, and improving competitiveness. In contrast, the evolutionary economics of innovation - in line with the networked model of innovation - stresses the systemic nature of innovation and thus advocates rectifying any systemic failure that hinders the generation, circulation, and exploitation of any type of knowledge required for successful innovation processes. This way of thinking has influenced the measurement and monitoring practices of the European Commission or the OECD to a significantly lesser extent than mainstream economics.
The IUS indicators could in principle be useful where the dominant mode of innovation is the ST mode. In practice, however, both the ST and DUI modes of innovation are important [Jensen et al., 2007]. Moreover, the so-called Summary Innovation Index - calculated from the IUS indicators - does not provide sufficient information to assess a given innovation system: its low value could reflect either an overall low level of innovation activities or a low level of R&D-based innovation activities (while other types of innovations are abundant). Nevertheless, that is an important distinction from an analytical as well as a practical (policy) point of view: these two innovation systems are fundamentally different, necessitating bespoke policy actions. Analysts and policy makers dealing with innovation, therefore, should pay attention to both R&D-based (ST) and non-R&D-based (DUI) innovations. Furthermore, while social innovations can certainly rely on R&D-based technological innovations, their essence tends to be organizational, managerial, and behavioural changes. The IUS indicators do not capture these types of changes. More generally, analysts and decision makers should be aware of the diversity of social innovations too in terms of their nature, drivers, objectives, actors, and process characteristics.
An assessment of the 81 indicators used to compile the Global Innovation Index has shown that it would not be fruitful to rely on any of the indicators to describe and characterize social innovations. The Technology Achievement Index, presented in the 2001 edition of the Human Development Report [UNDP, 2001], has not been discussed in this paper, but it is worth recalling that it does not offer a promising approach as it is not a comprehensive measure: it considers only certain types of technological achievements and not those that are the most relevant from the point of view of human development [Chiappero-Martinetti, 2015; Desai et al., 2002].
Some more general methodological lessons, however, can be distilled from the efforts devoted to measure business innovations. The first one concerns the use of composite indicators. Scoreboards and league tables compiled following the science-push logic, based on a composite indicator to establish rankings, and published by supranational organizations, can easily lead to 'lock-in' situations. National policy makers - and particularly politicians - are likely to pay much more attention to their country's position on a scoreboard than to nuanced assessments or policy recommendations in lengthy documents, and hence this inapt logic is 'diffused' and strengthened at the national level, too, preventing policy learning and devising appropriate policies. Despite the original intention to broaden the horizon of decision makers by offering internationally comparable data, these scoreboards and league tables strengthen a narrow-minded, simplifying approach.
In other words, given the diversity among innovation systems, one should be very careful when trying to draw policy lessons from the 'rank' of a country as 'measured' by a composite indicator. A scoreboard can only be constructed by using the same set of indicators across all countries, and by applying an identical method to calculate the composite index. Yet, it is important to realise that poor performance signalled by a composite indicator, and leading to a low rank on a certain scoreboard, does not automatically identify the area(s) necessitating the most urgent policy actions.
In contrast, a high rank on a scoreboard - such as Sweden's first place on the 2013 Innovation Union Scoreboard - does not necessarily reflect a satisfactory performance [European Commission, 2013a]. By taking into account the input and output nature of various IUS indicators, Edquist and Zabala-Iturriagagoitia [Edquist, Zabala-Iturriagagoitia, 2015] calculated the productivity of national innovation systems covered by the IUS. Using this assessment, which is undoubtedly highly relevant from a policy point of view, Sweden ranks a mere 24.
Analysts and policy makers, therefore, need to avoid the trap of paying too much attention to simplifying ranking exercises. Instead, it is of utmost importance to conduct detailed, thorough comparative analyses. It is crucial to identify the reasons for disappointing performance, as well as the sources of - and opportunities for — balanced, and sustainable, socio-economic development.
Second, the degree of novelty and the unit of analysis are interrelated issues when business innovations are surveyed. It seems a difficult task to establish the degree of novelty of a given social innovation. Actually, this issue seems to be of lesser importance in these cases: intellectual property rights are seldom an issue for social innovators. However, prestige - obtained by being acknowledged as a creative social innovator - might play a role. It could be perceived as an incentive to initiate social innovation projects. No doubt, it is an empirical question to establish the role of prestige in these endeavours. It could be also an interesting and demanding research question to identify whether a given social innovation is a stand-alone new solution or (using the analogy of technology systems) a part of a new 'social system.' A 'social system' could be a set of socially, institutionally, organizationally, and
economically interconnected social innovations that affect several groups of people or an entire community (a neighbourhood, village, town or city) at the same time. Occasionally, a 'social system' leads to the emergence of new social structures, norms, institutions, behaviour, value systems and practices at a higher level of aggregation such as at the level of sub-national regions, nations, or even supra-national regions (for example, the European Union).
Efforts aimed at measuring social innovation cannot rely on as long a tradition as those capturing technological innovation. The EC-funded 'Theoretical, Empirical and Policy Foundations for Social Innovation in Europe' (TEPSIE) project has contributed significantly to this goal. Although the proposed TEPSIE framework for measuring social innovation [Bund et al., 2013] has not been analysed here, it should be noted that its first pillar ('entrepreneurial activity') is not specific to social innovation. It also somewhat neglects non-entrepreneurial social innovation activities. Its second pillar ('field-specific output and outcomes') offers useful recommendations; yet we are faced by the usual attribution problem in the case of social innovations. The third pillar is concerned with framework conditions. The structure of the TEPSIE indicators prompts a more general caveat: analysts and policy makers need to be aware of the differences between measuring: (a) social innovation activities (or efforts) themselves; (b) the framework for social innovations (pre-requisites, available inputs, skills, norms, values, behavioural patterns, etc.); and (c) the economic, societal, or environmental impacts of social innovations.
This paper draws on work conducted for the project entitled 'Creating Economic Space for Social Innovation' (CrESSI) that has received funding from the European Union's Seventh Framework Programme for Research, Technological Development, and Demonstration (grant agreement No. 613261).
References
Arrow K.J. (1962) The Economic Implications of Learning by Doing. The Review of Economic Studies, vol. 29, no 3, pp. 155-173.
Balconi M., Brusoni S., Orsenigo L. (2010) In defence of the linear model: An essay. Research Policy, vol. 39, no 1, pp. 1-13.
Baumol W., Litan R., Schramm C. (2007) Good Capitalism, Bad Capitalism, and the Economics of Growth and Prosperity, New Haven: Yale University Press.
Baumol W.J. (2002) The Free-Market Innovation Machine: Analyzing the growth miracle of capitalism, Princeton: Princeton University Press.
Bender G., Jacobson D., Robertson P.L. (2005) Non-Research-Intensive Industries in the Knowledge Economy. Perspectives on Economic and Social Integration, vol. XI, no 1-2, pp. 49-73.
Bleda M., del Río P. (2013) The market failure and the systemic failure rationales in technological innovation systems. Research Policy, vol. 42, no 5, pp. 1039-1052.
Budd C.H., Naastepad C.W.M., Beers C. (eds.) (2015) Report on Institutions, Social Innovation & System Dynamics from the Perspective of the Marginalised (CRESSI Working Papers no 1/ 2015), Oxford: University of Oxford.
Bund E., Hubrich D-K., Schmitz B., Mildenberger G., Krlev G. (2013) Blueprint of social innovation metrics — Contributions to an understanding of opportunities and challenges of social innovation measurement (deliverable 2.4 of the TEPSIE project: The theoretical, empirical and policy foundations for building social innovation in Europe, European Commission 7th Framework Programme), Brussels: European Commission.
Bush V. (1945) Science: The Endless Frontier, Washington, D.C.: US Government Printing Office.
Caraba J., Lundvall B-A., Mendon^a S. (2009) The changing role of science in the innovation process: From Queen to Cinderella? Technological Forecasting and Social Change, vol. 76, no 6, pp. 861-867.
Castellacci F. (2008a) Innovation and the competitiveness of industries: Comparing the mainstream and the evolutionary approaches. Technological Forecasting and Social Change, vol. 75, no 7, pp. 984-1006.
Castellacci F. (2008b) Technological paradigms, regimes and trajectories: Manufacturing and service industries in a new taxonomy of sectoral patterns of innovation. Research Policy, vol. 37, no 6-7, pp. 978-994.
Chiappero-Martinetti E. (2015) Relationship between innovation/technology and human development (contribution to WP3 of the CrESSI project), Pavia: mimeo.
Cornell University, INSEAD, WIPO (2014) The Global Innovation Index 2014: The Human Factor in Innovation, Fontainebleau, Ithaca, Geneva: Cornell University, INSEAD, WIPO.
Desai M., Fukuda-Parr S., Johansson J., Sagasti F. (2002) Measuring Technology Achievement of Nations and the Capacity to Participate in the Network Age. Journal of Human Development, vol. 3, no 1, pp. 95-122.
Di Stefano G., Gambardella A., Verona G. (2012) Technology push and demand pull perspectives in innovation studies: Current findings and future research directions. Research Policy, vol. 41, no 8, pp. 1283-1295.
Dodgson M., Rothwell R. (eds.) (1994) The Handbook of Industrial Innovation, Cheltenham: Edward Elgar.
Dodgson M., Gann D.M., Phillips N. (eds.) (2014) The Oxford Handbook of Innovation Management, Oxford: Oxford University Press.
Dosi G. (1988a) The nature of the innovative process. Technical Change and Economic Theory (eds. G. Dosi, C. Freeman, R.R. Nelson, G. Silverberg, L. Soete), London: Pinter, pp. 221-238.
Dosi G. (1988b) Sources, procedures and microeconomic effects of innovation. Journal of Economic Literature, vol. 24, no 4, pp. 1120-1171.
Dosi G., Freeman C., Nelson R.R., Silverberg G., Soete L. (eds.) (1988) Technical Change and Economic Theory, London: Pinter.
Edquist C. (2011) Design of innovation policy through diagnostic analysis: Identification of systemic problems or (failures), Industrial and Corporate Change, vol. 20, no 6, pp. 1725-1753.
Edquist C. (ed.) (1997) Systems of Innovations: Technologies, institutions and organizations, London: Pinter.
Edquist C., Zabala-Iturriagagoitia J.M. (2015) The Innovation Union Scoreboard is Flawed: The case of Sweden — Not being the innovation leader of the EU (CIRCLE Papers in Innovation Studies, no 2015/16), Lund: Lund University.
Ergas H. (1986) Does Technology Policy Matter? Centre for European Policy Studies (CEPS Paper no 29), Brussels: European Commission.
Ergas H. (1987) The importance of technology policy. Economic Policy and Technological Performance (eds. P. Dasgupta, P. Stoneman), Cambridge, MA: Cambridge UP, pp. 51-96.
European Commission (2002) 2002 European Innovation Scoreboard (Technical Paper no 6, Methodology Report), Brussels: European Commission, Enterprise Directorate-General.
European Commission (2015) Innovation Union Scoreboard 2014, Brussels: European Commission, Directorate-General for Internal Market, Industry, Entrepreneurship and SMEs.
Eurostat (2012) Results of the community innovation survey 2012 (CIS2012), Paris: Eurostat. Available at: http:// ec.europa.eu/eurostat/cache/metadata/en/inn_cis8_esms.htm, accessed 23.07.2015.
Fagerberg J., Fosaas M., Bell M., Martin B. (2011) Christopher Freeman: Social Science Entrepreneur, Research Policy, vol. 40, no 7, pp. 897-916.
Fagerberg J., Landstrom H., Martin B. (2012) Innovation: Exploring the knowledge base. Research Policy, vol. 41, no 7, pp. 1132-1153.
Fagerberg J., Mowery D.C., Nelson R.R. (eds.) (2005) The Oxford Handbook of Innovation, Oxford: Oxford University Press.
Foray D. (ed.) (2009) The New Economics of Technology Policy, Cheltenham: Edward Elgar.
Freeman C. (1991) Networks of innovators, a synthesis of research issues. Research Policy, vol. 20, no 5, pp. 499-514.
Freeman C. (1994) The economics of technical change: A critical survey. Cambridge Journal of Economics, vol. 18, no 5, pp. 463-514.
Freeman C. (1995) The "National System of Innovation" in historical perspective. Cambridge Journal of Economics, vol. 19, no 1, pp. 5-24.
Freeman C., Soete L. (1997) The Economics of Industrial Innovation (3rd edition), London: Pinter.
Godin B. (2006) The Linear Model of Innovation: The Historical Construction of an Analytical Framework. Science, Technology & Human Values, vol. 31, no 6, pp. 639-667.
Godin B. (2008) The moral economy of technology indicators. Innovation in Low-Tech Firms and Industries (eds. H. Hirsch-Kreinsen, D. Jacobson), Cheltenham: Edward Elgar, pp. 64-84.
Grupp H. (1998) Foundations of the Economics of Innovation: Theory, measurement and practice, Cheltenham: Edward Elgar.
Grupp H., Schubert T. (2010) Review and new evidence on composite innovation indicators for evaluating national performance. Research Policy, vol. 39, no 1, pp. 67-78.
Hall B.H., Rosenberg N. (eds.) (2010) Economics of Innovation, Amsterdam: North-Holland.
Hatzichronoglou T. (1997) Revision of the High-Technology Sector and Product Classification (OECD STI Working Papers, 1997/2), Paris: OECD.
Havas A. (2015a) The persistent high-tech myth in the EC policy circles: Implications for the EU10 countries (Institute of Economics CERS HAS Discussion Papers MT-DP - 2015/17), Budapest: Hungarian Academy of Sciences.
Havas A. (2015b) How does social innovation challenge neo-classical assumptions regarding technological innovation? Report Contrasting CrESSI's Approach of Social Innovation with that of Neoclassical Economics (eds. C. Houghton Budd, C.W.M. Naastepad, C.P. van Beers) (CrESSI Report D1.3), Oxford: Oxford University, pp. 40-46.
Havas A. (2015c) Various approaches to measuring business innovation: their relevance for capturing social innovation (Institute of Economics CERS HAS Discussion Papers MT-DP - 2015/54), Budapest: Hungarian Academy of Sciences.
Hirsch-Kreinsen H., Jacobson D. (eds.) (2008) Innovation in Low-Tech Firms and Industries, Cheltenham: Edward Elgar.
Hirsch-Kreinsen H., Jacobson D., Laestadius S. (eds.) (2005) Low Tech Innovation in the Knowledge Economy, Frankfurt: Peter Lang.
Hirsch-Kreinsen H., Jacobson D., Laestadius S., Smith K. (2005) Low and Medium Technology Industries in the Knowledge Economies: The Analytical Issues. Low Tech Innovation in the Knowledge Economy (eds. H. Hirsch-Kreinsen, D. Jacobson, S. Laestadius), Frankfurt: Peter Lang, pp. 11-29.
Hirsch-Kreinsen H., Schwinge I. (eds.) (2014) Knowledge-intensive Entrepreneurship in Low-Tech Industries, Cheltenham: Edward Elgar.
Hollanders H., Tarantola S. (2011) Innovation Union Scoreboard 2010 - Methodology report, Maastricht, Brussels: UNU-MERIT, European Commission.
Jensen M.B., Johnson B., Lorenz E., Lundvall B-A. (2007) Forms of knowledge and modes of innovation. Research Policy, vol. 36, no 5, pp. 680-693.
Kaloudis A., Sandven T., Smith K. (2005) Structural change, growth and innovation: The roles of medium and low-tech industries, 1980-2000, Low Tech Innovation in the Knowledge Economy (eds. H. Hirsch-Kreinsen, D. Jacobson, S. Laestadius), Frankfurt: Peter Lang, pp. 31-63.
Klevorick A.K., Levin R.C., Nelson R.R., Winter S.G. (1995) On the sources and significance of interindustry differences in technical opportunities. Research Policy, vol. 24, no 2, pp. 185-205.
Kline S.J., Rosenberg N. (1986) An Overview of Innovation. The Positive Sum Strategy: Harnessing Technology for Economic Growth (eds. R. Landau, N. Rosenberg), Washington: National Academy Press, pp. 275-305.
Laestadious S., Pedersen T.E., Sandven T. (2005) Towards a new understanding of innovativeness - and of innovation based indicators, Low Tech Innovation in the Knowledge Economy (eds. H. Hirsch-Kreinsen, D. Jacobson, S. Laestadius), Frankfurt: Peter Lang, pp. 75-121.
Lazonick W. (2013) The Theory of Innovative Enterprise: Methodology, Ideology, and Institutions. Alternative Theories of Competition: Challenges to the Orthodox (eds. J.K. Moudud, C. Bina, P.L. Mason), London: Routledge, pp. 127-159.
Lundvall B-A. (ed.) (1992) National Systems of Innovation: Towards a Theory of Innovation and Interactive Learning, London: Pinter.
Lundvall B-A., Borrás S. (1999) The Globalising Learning Economy: Implications for Innovation Policy, Luxembourg: Office for Official Publications of the European Communities.
Malerba F. (2002) Sectoral systems of innovation and production. Research Policy, vol. 31, no 2, pp. 247-264.
Martin B. (2012) The evolution of science policy and innovation studies. Research Policy, vol. 41, no 7, pp. 1219-1239.
Mendon^a S. (2009) Brave old world: Accounting for 'high-tech' knowledge in 'low-tech' industries. Research Policy, vol. 38, no 3, pp. 470-482.
Metcalfe S. (1998) Evolutionary Economics and Creative Destruction, London: Routledge.
Mowery D.C. (2009) Plus 9a change: Industrial R&D in the "third industrial revolution". Industrial and Corporate Change, vol. 18, no 1, pp. 1-50.
Mowery D.C., Nelson R.R. (eds.) (1999) Sources of Industrial Leadership: Studies of Seven Industries, Cambridge: Cambridge University Press.
Nelson R.R. (1995) Recent evolutionary theorizing about economic change. Journal of Economic Literature, vol. 33, no 1, pp. 48-90.
Nelson R.R. (ed.) (1993) National Innovation Systems: A comparative study, Oxford: Oxford University Press.
OECD (1992) TEP: The Key Relationships, Paris: OECD.
OECD (1998) New Rationale and Approaches in Technology and Innovation Policy (STI Review, no 22), Paris: OECD.
OECD (2001) Innovative Networks: Co-operation in national innovation systems, Paris: OECD.
OECD (2002) Frascati Manual: Proposed Standard Practice for Surveys on Research and Experimental Development (6th edition), Paris: OECD.
OECD (2005) Oslo Manual: Guidelines for Collecting and Interpreting Innovation Data (3rd edition), Paris: OECD.
Pavitt K. (1984) Sectoral patterns of technical change: Towards a taxonomy and theory. Research Policy, vol. 13, no 6, pp. 343-373.
Pavitt K. (1999) Technology, Management and Systems of Innovation, Cheltenham: Edward Elgar.
Peneder M. (1999) The Austrian Paradox: "Old" structures but high performance? Austrian Economic Quarterly, vol. 4, no 4, pp. 239-247.
Peneder M. (2010) Technological regimes and the variety of innovation behaviour: Creating integrated taxonomies of firms and sectors. Research Policy, vol. 39, no 3, pp. 323-334.
Perez C. (2010) Technological revolutions and techno-economic paradigms, Cambridge Journal of Economics, vol. 34, no 1, pp. 185-202.
Robertson P., Smith K. (2008) Distributed knowledge bases in low- and medium technology industries. Innovation in Low-Tech Firms and Industries (eds. H. Hirsch-Kreinsen, D. Jacobson), Cheltenham: Edward Elgar, pp. 93-117.
Robertson P., Smith K., von Tunzelmann N. (2009) Innovation in low- and medium-technology industries. Research Policy, vol. 38, no 3, pp. 441-446.
Rosenberg N. (1982) Inside the black box: Technology and economics, Cambridge: Cambridge University Press.
Rosenberg N., Landau R. (eds.) (1986) The Positive Sum Strategy: Harnessing Technology for Economic Growth, Washington, D.C.: National Academies Press.
Rosenberg N., Mowery D.C. (1998) Paths of Innovation: Technological Change in 20th-century America, New York: Cambridge University Press.
Smith K. (2000) Innovation as a Systemic Phenomenon: Rethinking the Role of Policy. Enterprise & Innovation Management Studies, vol. 1, no 1, pp. 73-102.
Smith K. (2002) What is the "Knowledge Economy"? Knowledge intensity and distributed knowledge bases (UNU/ INTECH Discussion Paper Series, 2002-6), Maastricht: United Nations University.
Smith K. (2005) Measuring Innovation. The Oxford Handbook of Innovation (eds. J. Fagerberg, D.C. Mowery, R.R. Nelson), Oxford: Oxford University Press, pp. 148-177.
Tidd J., Bessant J., Pavitt K. (1997) Managing Innovation: Integrating technological, market and organizational change, Chichester: John Wiley & Sons.
UNDP (2001) Human Development Report: Making new technologies work for human development, Oxford: Oxford University Press.
von Hippel E. (1988) The Sources of Innovation, Oxford: Oxford University Press.
von Tunzelmann N. (1995) Technology and Industrial Progress: The foundations of economic growth, Aldershot: Edward Elgar.
von Tunzelmann N., Acha V. (2005) Innovation in "Low-Tech" Industries. The Oxford Handbook of Innovation (eds. J. Fagerberg, D.C. Mowery, R.R. Nelson), Oxford: Oxford University Press, pp. 407-432.
