Научная статья на тему 'The Automation of Jobs: A Threat for Employment or a Source of New Entrepreneurial Opportunities?'

The Automation of Jobs: A Threat for Employment or a Source of New Entrepreneurial Opportunities? Текст научной статьи по специальности «Экономика и бизнес»

CC BY-NC-ND
4679
600
i Надоели баннеры? Вы всегда можете отключить рекламу.
Журнал
Форсайт
Scopus
ВАК
RSCI
ESCI
Область наук
Ключевые слова
automation / jobs / occupational mobility / entrepreneurship / digitalization.

Аннотация научной статьи по экономике и бизнесу, автор научной работы — Alina Sorgner

New and emerging technologies pose a serious challenge for the future of employment. As machines learn to accomplish increasingly complex production tasks, the concern arises that automation will wipe out a great number of jobs. This paper investigates the relationship between the risk posed by the automation of jobs and individual-level occupational mobility using a representative German household survey. It provides an overview of current trends and developments on the labor markets due to the automation of jobs. It also describes the most recent dynamics of self-employment and relates it to the risk of the automation of jobs. The results suggest that the expected occupational changes such as losing a job, demotion at one’s current place of employment, or starting a job in a new field are likely to be driven by the high occupation-specific risk of automation. However, the switch to self-employment, both with and without employees, is more likely to occur from paid employment in occupations with a low risk of automation. Hence, the rising level of entrepreneurial activities is less likely due to jobs becoming obsolete over the course of automation, but rather due to the high number of opportunities offered by the digital age. The issues addressed within the paper provide room for further investigation. Particularly, an important question concerns which additional skills workers in jobs with a high risk of automation should acquire in order to make themselves less susceptible to the negative consequences of such a change. More research is needed in order to develop educational strategies to make workers less susceptible to job loss due to automation.

i Надоели баннеры? Вы всегда можете отключить рекламу.
iНе можете найти то, что вам нужно? Попробуйте сервис подбора литературы.
i Надоели баннеры? Вы всегда можете отключить рекламу.

Текст научной работы на тему «The Automation of Jobs: A Threat for Employment or a Source of New Entrepreneurial Opportunities?»

The Automation of Jobs: A Threat for Employment or a Source of New Entrepreneurial Opportunities?

Alina Sorgner a;b

Researcher, alina.sorgner@uni-jena.de

a School of Economics and Business Administration, Friedrich Schiller University Jena, Carl-Zeiss-Str. 3, 07743 Jena, Germany.

b Kiel Institute for the World Economy, Kiellinie 66 D-24105 Kiel, Germany.

Abstract

New and emerging technologies pose a serious challenge for the future of employment. As machines learn to accomplish increasingly complex production tasks, the concern arises that automation will wipe out a great number of jobs. This paper investigates the relationship between the risk posed by the automation of jobs and individual-level occupational mobility using a representative German household survey. It provides an overview of current trends and developments on the labor markets due to the automation of jobs. It also describes the most recent dynamics of self-employment and relates it to the risk of the automation of jobs.

The results suggest that the expected occupational changes such as losing a job, demotion at one's current place of employment, or starting a job in a new field are

likely to be driven by the high occupation-specific risk of automation. However, the switch to self-employment, both with and without employees, is more likely to occur from paid employment in occupations with a low risk of automation. Hence, the rising level of entrepreneurial activities is less likely due to jobs becoming obsolete over the course of automation, but rather due to the high number of opportunities offered by the digital age.

The issues addressed within the paper provide room for further investigation. Particularly, an important question concerns which additional skills workers in jobs with a high risk of automation should acquire in order to make themselves less susceptible to the negative consequences of such a change. More research is needed in order to develop educational strategies to make workers less susceptible to job loss due to automation.

Keywords:

automation; jobs;

occupational mobility;

entrepreneurship;

digitalization.

Citation: Sorgner A. (2017) The Automation of Jobs: A Threat for Employment or a Source of New Entrepreneurial Opportunities? Foresight and STI Governance, vol. 11, no 3, pp. 37-48. DOI: 10.17323/25002597.2017.3.37.48

Recent technological progress, particularly in the field of ICT, has led to the emergence of Industry

4.0, the fourth industrial revolution, and has given rise to a debate about the future of employment.

There is great concern that as technology develops further, it will become possible for machines to perform tasks at least as efficiently as the humans who are currently performing them. As a consequence, it is feared that automation will lead to a massive wipe-out of jobs. Researchers from the University of Oxford, Carl Benedikt Frey and Michael Osborne, recently arrived at the conclusion that, given the current state of technology, about 47 percent of the US labor force are in jobs that are highly likely to be replaced by machines in the next 10-20 years [Frey, Osborne, 2013, 2017]. Numerous follow-up studies generally confirmed this scenario for other countries, though they report great variations in automation risk across countries.1

The aforementioned studies, however, provide estimates based on aggregate employment data and it therefore remains unclear whether or not and how far the predicted risk of job automation2 is associated with occupational mobility at the individual level. Hence, the present paper aims to shedd more light on this relationship by investigating whether working in an occupation with a high risk of automation affects job changes, such as the risk of losing one's job, demotion at one's current place of employment, or starting a job in a new field, among others. Then, this paper investigates the impact of the automation of jobs on the probability of becoming an entrepreneur. The recent rise of entrepreneurship in many developed countries that has been observed over the last two decades raises many questions with regard to the drivers of this development and the quality of those start-ups. Particularly, the technological progress that leads to the automation of jobs may foster start-ups created by necessity by those people whose jobs are likely to be replaced with machines. At the same time, technological progress may lead to an increase in the level of opportunity-driven and growth-oriented start-ups. For instance, Shane [Shane, 2000] discusses how the introduction of a radical innovation such as 3D printing technology has led to the emergence of entrepreneurial opportunities in very different fields ranging from the creation of personalized sculptures from a photo to constructing prototype models of industrial and architectural design to printing out artificial bones and creating three-dimensional human brain models for surgical planning. Obviously, some of those opportunities emerge in fields (e.g., surgery) in which machines complement rather than substitute human labor.

The present empirical analysis is based on German Socio-Economic Panel data, an annual representative survey of German households containing rich information on an individual's socioeconomic background. The results suggest that the risk of the automation of jobs increases the risk of occupational changes such as losing a job, demotion at one's current place of employment, or starting a job in a new field within the next two years. At the same time, the risk of automation is negatively correlated with the probability of becoming an entrepreneur, both with and without employees. This suggests that entrepreneurs are less likely to be driven by necessity arising from the risk of job automation. Thus, the rising level of entrepreneurship in Germany is more likely to be driven by new technology creating new entrepreneurial opportunities rather than by destroying jobs.

The Labor Market in the Digital Age: Trends and Developments

This section reviews trends and developments that are currently present on the labor markets of many developed countries and are related to the current progress in the automation of tasks. In particular, we describe the phenomenon of the polarization of labor markets. Then, we discuss the consequences of automation for the future of employment by considering current trends in the rate of entrepreneurial activities.

Which jobs are at risk of automation?

In order to understand which jobs are at high risk of automation, it is necessary to analyze what types of tasks can be effectively and efficiently performed by computers and those tasks in which computers merely supplement human labor. The authors of [Autor et al., 2003] distinguish between two broad sets of tasks according to the extent of their vulnerability to computerization, namely, routine and non-routine tasks. The group of non-routine tasks can be further divided into manual and abstract tasks. Due to the nature of routine tasks that may be both cognitive (e.g. performing calculations) and physical (e.g., repetitive operations in a stable environment), they can be fully codified and thus jobs that mainly comprise routine tasks are highly susceptible to computerization. While machines outperform humans in many of the routine tasks, they have not yet achieved high performance levels when carrying out non-routine tasks, that is, manual and abstract tasks. Manual tasks are those activities that can be easily performed by humans but require enormous computing power from machines. Examples of such tasks are manual operations in unstable, changing environments that require high adaptability and manual dexterity as well as visual and language recognition. One should note, however, that the current progress in artificial intelligence (AI) is

1 See, e.g., [Arntz et al., 2016], for an analysis of OECD countries; [Sorgner et al., 2017a] for G20 countries; [Chang, Huynh, 2016], for ASEAN countries; [Dengler, Matthes, 2015], for the case of Germany; [Pajarinen et al., 2015], for an analysis of Finland and Norway).

2 In this paper the terms computerization, digitization, and automation are used interchangeably. 38 | FORESIGHT AND STI GOVERNANCE | Vol. 11 No 3 | 2017

quite impressive and it can be expected that machines will learn to perform those tasks even better in the near future (see [Brynjolfsson, McAfee, 2014], for other examples). Still, humans currently perform these tasks at a much lower cost, which is why there is a relatively low risk of computerization for such jobs. Last but not least, abstract tasks require creativity, persuasion, and problem-solving skills, in such capacities computers tend to complement highly educated workers.

Given this state of technology, the major trend that is currently observed in various developed countries is that of the polarization of labor markets [Autor, 2015a; Autor, Dorn, 2013; Goos et al., 2014]. Job polarization is a phenomenon that refers to the growth of employment at opposite ends of the occupational skill spectrum. That is, there has been an increase in highly paid jobs that require high levels of education and are mostly comprised of abstract tasks on the one hand, and on the other, there is growth in low-pay jobs that comprise manual tasks performed by people with lower levels of education.

Recently, in a study of the future of employment in the US, Frey and Osborne [Frey, Osborne, 2013, 2017] arrived at the conclusion that about 47 percent of the US labor force is currently working in occupations with a particularly high risk of being computerized in the next 10-20 years. Those high-risk occupations mainly comprise transportation and logistics occupations, office and administrative support workers, and production occupations. The OECD's Directorate for Employment, Labour and Social Affairs commissioned a similar study for OECD countries [Arntz et al., 2016], whose authors concluded that the risk of automation might have been overestimated. On contrary, they find that, on average across the 21 OECD countries, only about 9% of jobs are automatable, although there is great variation between the countries. The highest risk of automation was found for Germany and Austria (12%) and the lowest for Korea and Estonia (6%). The study by Chang and Huynh [Chang, Huynh, 2016] of ASEAN countries, however, reports that about 56 percent of employment is at high risk of displacement over the next decade or two. And the study by Sorgner, Bode, and Krieger-Boden [Sorgner et al., 2017a] of G20 countries provides evidence of gender-specific differences in the effects of automation on employment, which also vary strongly across countries. This strong crosscountry variation may reflect general differences in the industrial structures of the economies in those countries. For instance, knowledge-intensive sectors have jobs that heavily rely upon abstract tasks, while many jobs in manufacturing sectors are routine-based and, thus, susceptible to automation.

While the study by Frey and Osborne [Frey, Osborne, 2013, 2017] for the US and similar studies for other countries focused on aggregated employment data, it is not clear how the risk of automation and computerization affects occupational changes at the level of individuals, such as transitions from paid employment into unemployment or self-employment. Hence, this paper's aim is to shed more light on this issue by investigating micro-level data.

The automation of jobs and the rise of entrepreneurship

A pronounced development that many developed countries are currently experiencing is that of a fundamental shift from a managed economy to an entrepreneurial economy. The term 'managed economy' refers to the organization of market economies after WWII that were characterized by the prevalence of economies of scale, standard production routines, high levels of specialization and relatively low levels of uncertainty in the manufacturing process. In contrast to the managed economy, the entrepreneurial economy is predominantly based on pronounced start-up activity, innovation that occurs at entrepreneurial organizations, flexible production and flexible labor markets, and relatively high levels of uncertainty [Audretsch, Thurik, 2000, 2001]. Moreover, new business formation, while being largely neglected by policy makers in the managed economies — starts to play an increasingly important role in the entrepreneurial economy with regard to its direct effects such as job creation [Acs, 2011] and, more importantly, the indirect effects. Concerning the latter, start-ups represent an important challenge for incumbent firms and, thus, force them to perform more efficiently [Fritsch, 2011]. Last but not least, new entrants may create new markets by introducing radical innovations [Baumol, 2004].

This shift towards a more entrepreneurial economy is well reflected in Figure 1, which shows the dynamics of self-employment rates in Germany during the period between 1991 and 2012. The self-employment rate grew steadily from about 8 percent in the beginning of the observation period up to about 11.5 percent at the end of the period. It is worth noting that this development cannot be entirely attributed to the former GDR's transition to a market economy, although the event of German reunification contributed significantly to the rise of the overall self-employment rate in Germany. Particularly, the self-employment rate in East Germany converged with the level of self-employment activities in West Germany around 2004 and even exceeded it thereafter. Nevertheless, the rise of entrepreneurial activities can also be observed in West German regions.

This evidence leads to a question of what the drivers of this fundamental move towards a more entrepreneurial society are. The rising level of entrepreneurial activities may simply reflect various structural changes in a society. For instance, changes in the socio-demographic characteristics of the population, such as the age structure, the increased rate of female participation in the labor market, and an average higher level of education, might have led to a rise in entrepreneurship [Fritsch et al., 2015].3 Moreover, the incentives

3 This is because socio-demographic characteristics are important determinants for making the decision to become an entrepreneur [Parker, 2009].

Figure 1. Dynamics of self-employment rates in Germany, 1991-2012

c^^^^c^^^^^ooooooooooooo

East Germany (including Berlin) West Germany Self-employment rate in Germany

Self-employment rate in West Germany — Self-employment rate in East Germany

Source: own calculations based on the German Micro-Census.

to become an entrepreneur may have changed as a response to a variety of policy measures designed to promote entrepreneurship that have been realized over approximately the last two decades. For example, in Germany, a whole range of public policy measures have been realized, which promoted, for instance, start-ups by unemployed persons [Caliendo, Kritikos, 2010] and by students as well as highly educated staff at universities and other public research institutes ('EXIST'). Some of those programs aim to reduce start-up barriers for women related to human and financial capital for women [Welter, 2006].4 Those policy measures may have at least partially shaped a more pro-entrepreneurial attitude among the population and a stronger awareness of entrepreneurship as an alternative career option thus contributing to the growth of entrepreneurship. Last but not least, technological progress, in particular achievements in ICT that gave rise to the fourth industrial revolution, may be responsible for a great number of entrepreneurial opportunities and the shift towards an entrepreneurial economy in many developed countries [Audretsch, Thurik, 2000]. Remarkably, Figure 2 demonstrates that the rise of self-employment rates in Germany was predominantly due to the rise in self-employment without employees (solo self-employment). While the solo self-employment rate increased from about 3.5 percent in 1991 to 6 percent in 2012, the level of self-employment with employees increased only negligibly. The businesses of the solo self-employed have often been regarded as secondary (low quality) start-ups, as they generally do not create much value in terms of innovation, employment growth, and wealth generation [Shane, 2009].

This recent rise in the level of self-employment is quite remarkable and requires investigation, as there is not much evidence concerning the reasons behind this development and about the quality of such startups. Fritsch et al. [Fritsch et al., 2015] conducted a decomposition analysis of self-employment rates in Germany over time in order to determine the major drivers of such a pronounced, radical change. They provide evidence that demographic developments, such as the shift towards employment in the service sector and a larger share of the population holding tertiary degrees, are the major drivers of the increase in the overall level of self-employment. While these factors explain most of the developments in self-employment with employees, they could only explain less than a half of the much more dramatic increase in self-employment without employees. In particular, it remains unclear whether or not and how far the rise of solo self-employment was triggered by technological progress that has led to the automation of jobs. It can be assumed that people in jobs with a relatively high risk of automation may be more likely to set up businesses out of necessity. Furthermore, it can be expected that such businesses are less likely to be growth-oriented, as they have been created with the primary aim of employing the business owner

4 Other policy measures include, for instance, the deregulation of a number of liberal occupations, the introduction of minimum capital requirements for setting up a limited liability company, changes to personal bankruptcy law, and more restricted access to social security support.

Figure 2. Dynamics of self-employment with and without employees in Germany, 1991-2012

5000 4500 4000 3500 3000

Thonsend Share of total population (%)

„."•.......

............

2500 __

2000

1500 1000 500 0

Self-employed with employees ~~ Self-employment rate, with employees Source: own calculations based on the German Micro-Census.

Self-employed without employees Self-employment rate, without employees

[Shane, 2009]. Thus, the automation of human labor may indeed drive the levels of solo self-employment "out of necessity". On the contrary, technological progress may lead to a rise in opportunity-driven, growth-oriented entrepreneurship in fields that are less susceptible to automation and are characterized by creative and abstract tasks. Hence, another aim of this paper is to investigate the relationship between the risk of job automation and an individual's likelihood of becoming an entrepreneur.

Data and method

Data source

The empirical analysis is based on German Socio-Economic Panel (SOEP) data, which is an annual representative household survey conducted by the German Institute for Economic Research (DIW). It includes detailed information about the socioeconomic situations of approximately 22,000 individuals living in Germany [Wagner et al., 2007]. For the purposes of the present analysis, data for the time period from 2005 to 2013 was used.

Dependent variables

Currently employed individuals are asked in the SOEP to assess the probability of occupational changes over the next two years on a 10-point Likert scale from 0 to 100 with 10-point steps. The questions comprise various types of occupational changes related to losing or switching a job, an occupational promotion or demotion, expected wage/salary increases, etc. Many of those changes might occur as a consequence of the progress of computerization and/or the automation of jobs. The precise wording of questions used in the present study is as follows:

"How likely is it that you will experience the following career changes within the next two years? Please estimate the probability on a scale of 0 to 100, with 0 meaning that such a change definitely will not take place, and 100 meaning that such a change definitely will take place."

• "Will you lose your job?"

• "Will you stop working in your current field and start working in a different one?"

• "Will you be demoted at your current place of employment?"

• "Will you attend courses or seminars to obtain additional training or qualifications?"

• "Will you receive a salary or wage increase beyond the collectively negotiated wage increases?"

• "Will you start working on a self-employed and/or freelance basis?"

In addition to the anticipated occupational changes, in our further analysis we study the actual transitions from paid employment to unemployment and self-employment. To this end, two dependent variables were constructed as binary variables that equal to one if a respondent's observed employment status in time period t is paid employment and his or her employment status two years later is unemployment (self-

7

6

5

4

3

2

1

0

employment). In addition, for respondents who switched from paid employment to self-employment, we distinguish between those who did so on a solo basis and those who are self-employed with employees, in order to partially account for necessity- and opportunity-based motives.5

Independent variable

The variable of interest is the occupation-specific probability of automation, which indicates the level of risk for a particular occupation to be automated or computerized in the next one to two decades. This variable is based upon and was adapted from the study by Frey and Osborne [Frey, Osborne, 2013, 2017], who estimated the automation/computerization probabilities for 702 occupations according to the US occupational classification system, O*Net. Together with a group of experts in machine learning and robotics, Frey and Osborne were able to identify a set of occupations that they labeled with 1, meaning a 100 percent probability of the occupation being computerized in the next one to two decades, or 0 if the risk of computerization was considered absent.6 In the next step, they identified technological bottlenecks for computerization, that is, occupation-specific tasks that represent a challenge for machines. In particular, they identified three types of such bottlenecks, namely, social intelligence, creativity, and manipulation (perception). While social intelligence and creativity require high abilities and represent abstract tasks, manipulation and perception are mostly manual tasks (such as manual dexterity or tasks that are related to an unstructured work environment) that can be easily performed by humans, but represent a significant challenge for robots and machines.7 Finally, on the basis of the labeled data set, the researchers developed the optimal predicting algorithm, which they then used to estimate the probabilities of computerization for the remaining occupations based on the extent to which they are composed of tasks that pose bottlenecks to computerization.

Frey and Osborne [Frey, Osborne, 2013, 2017] provide the estimated probabilities of computerization for the 6-digit U.S. System of Occupational Classification (2010 SOC). Hence, they need to be converted to the 4-digit ISCO88 occupations that are available in SOEP in order to match them with other individual-level information. For this purpose, the algorithm created by the US Bureau of Labor Statistics was used.

Control variables

A wide set of control variables is considered, which can affect an individual's occupational mobility. In particular, information is available about the number of years that a respondent spent in formal education, tenure with his or her current employer, experienced years of unemployment, socio-demographic characteristics including age, gender, nationality, and children in the household. Moreover, SOEP data include short item scales that measure the Big Five dimensions of the personality [Costa, McCrae, 1992]. In particular, the survey includes 15 items, three for each of the five traits, which has been shown to accurately replicate the results of the more extensive 25-item Big Five inventory [Gerlitz, Schupp, 2005]. Psychological personality characteristics may affect an individual's willingness to change occupations in general, for instance, if a person has strong preferences for variety in his or her occupational environment [Astebro, Thompson, 2011]. Another reason for including personality traits in the model is that they may, to a certain extent, capture unobserved abilities beyond those measured by the level of formal education that may affect both the choice of a certain occupation and the probability of occupational changes. For a definition of the variables used in the analysis and how certain traits were measured see Table 1, which provides information on the descriptive statistics for independent variables. According to this table, the average respondent in the sample is about 42 years old and has enjoyed about 12.2 years of formal education. The average respondent has been at his or her current job for about 11 years and has experienced 0.9 years of unemployment in the past.

Method

The regression method used in the empirical analysis of expected occupational changes accounts for the peculiarities of the dependent variable, which is the probability of an occupational change occurring within the next two years. Since the dependent variable is bounded between zero and one, the model can be estimated by means of a fractional response model (FRM) proposed by Papke and Wooldridge [Papke, Wooldridge, 1996].

The analysis of transitions from paid employment to unemployment and self-employment is conducted by means of a probit regression, which accounts for the binary nature of the dependent variable that takes a value of 1 if an occupational change has occurred and takes a value of 0 if this was not the case.

5 It should be noted, however, that not all solo self-employed started their businesses out of necessity. For instance, Sorgner et al. [Sorgner et al., 2017b] show that while the solo self-employed tend to have lower incomes than comparable entrepreneurs with paid employees, there are entrepreneurial "super-stars" among the solo self-employed who earn more. The data do not allow for making a more precise distinction between the motives for a start-up.

6 Examples of occupations that they labeled with 1 are legal assistants or fast food cooks. Examples of occupations with a 0, no risk of computerization, are housekeepers, taxi drivers or economists.

7 This phenomenon is also known as Polanyi's paradox, which refers to the inability to convert tacit knowledge into codifiable and, hence, computerizable rules [Autor, 2015b].

Table 1. Descriptive statistics for independent variables

Variable Definition and measurement Mean Standard deviation

Occupation-specific probability ofautomation The risk of a certain occupation being computerized in the next 10-20 years. Adapted from [Frey, Osborne, 2013, 2017] 0.471 0.313

Years of formal education Number of years spent in formal education 12.219 2.700

Tenure, years Number of years spent at the current place of employment 11.016 10.367

Experience of unemployment, years Number of years spent in unemployment 0.917 2.307

Age Respondent's age 41.971 22.368

Gender Dummy variable indicating the gender of a respondent (=1 if male, =0 if female) 0.489 0.500

Nationality Dummy variable indicating the nationality of a respondent (=1 if German, =0 otherwise) 0.715 0.451

Presence of children in the household Dummy variable indicating if there are children in the household (=1 if yes, =0 if no) 0.393 0.488

Willingness to take risks Likert-type scale from 0 ("completely unwilling to take risks" to 10 ("fully willing to take risks") Precise question: "Are you generally a person who is fully prepared to take risks or do you try to avoid taking risks?" 4.564 2.329

Conscientiousness Average score from the scores on 3 questions (7-point Likert-type scale) measuring the Big Five dimension "Conscientiousness" 5.827 0.961

Extraversion Average score from the scores on 3 questions (7-point Likert-type scale) measuring the Big Five dimension "Extraversion 4.818 1.140

iНе можете найти то, что вам нужно? Попробуйте сервис подбора литературы.

Agreeableness Average score from the scores on 3 questions (7-point Likert-type scale) measuring the Big Five dimension "Agreeableness" 5.398 0.978

Openness to experience Average score from the scores on 3 questions (7-point Likert-type scale) measuring the Big Five dimension "Openness to experience" 4.485 1.212

Neuroticism (reversed emotional stability) Average score from the scores on 3 questions (7-point Likert-type scale) measuring the Big Five dimension "Degree of emotional stability" (emotional stability vs. neuroticism) 3.865 1.228

Internal locus of control Average score on two 7-point Likert scales measuring the internal locus of control. Precise statements: "How my life goes depends on me" and "One has to work hard in order to succeed" 5.737 0.954

External locus of control Average score on six 7-point Likert scales measuring external locus of control. Precise questions: 1) What a person achieves in life is above all a question of fate or luck; 2) Compared to other people, I have not achieved what I deserve; 3) I frequently have the experience that other people have a controlling influence over my life; 4) The opportunities that I have in life are determined by social conditions; 5) Innate abilities are more important than any efforts that one can make; 6) I have little control over the things that happen in my life. 3.683 0.940

Source: author's calculations.

Descriptive results

According to Table 2, the highest average probability of an occupational change to occur within the next two years reported by currently employed respondents was related to the risk being demoted at one's current place of employment (about 46 percent), followed by the probability of acquiring additional qualifications (about 39 percent) and losing a job (about 21 percent). Interestingly, the lowest average probability of occupational change is that due to a shift to self-employment (about 8 percent). Nevertheless, it should be noted that this number is about 8 times higher than the yearly start-up rate in Germany, which is only about 1 percent [Fritsch et al, 2012]. This indicates a rather high willingness among the German population to set up a business, a potential event that apparently could not be realized to its full extent.8

Table 3 presents the probability of occupational changes for respondents who are in occupations that are differently affected by the risk of automation. In particular, we distinguish between three groups of automation risk: low (less than 30 percent), medium (30 to 70 percent) and high (more than 70 percent). The descriptive evidence in Table 3 suggests that the probability of occupational changes increases with the rising risk of computerization for the categories of losing a job, starting work in a new field, and demotion at a current job. Moreover, a higher risk of the computerization of an occupation is associated with an on average lower probability of acquiring additional qualifications. The same pattern was observed for the probability of becoming self-employed. Last but not least, there seems to be a non-linear relationship between the risk of automation and the probability of an increase in wages. However, this relationship may be determined by other factors, such as previous labor market experience or the level of formal education,

8 The reasons for this may be manifold, such as a lack of financial capital, the inability to identify a profitable entrepreneurial opportunity, or the decreasing level of over-optimism over the business's gestation period.

Table 2. Descriptive statistics for dependent variables (the probability of occupational changes within the next 2 years)

Questions on the probability of occupational changes Mean (%) Standard deviation

Will you lose your job? 20.79 25.58

Will you stop working in your current field and start working in a different one? 12.17 23.34

Will you be demoted at your current place of employment? 46.28 14.88

Will you attend courses or seminars to obtain additional training or qualifications? 38.64 36.12

Will you receive a salary or wage increase beyond the collectively negotiated wage increases? 15.80 25.64

Will you start working on a self-employed and/or freelance basis? 7.94 20.44

Source: author's calculations.

among others. Hence, this relationship will be investigated in the next section in a multivariate analysis, in which we control for a wide set of socio-demographic characteristics that may influence the result.

The Results of Multivariate Analysis

The risk of computerization and perceived occupational changes

The results of a multivariate analysis of the relationship between the occupation-specific probability of computerization and the self-reported probability of occupational changes within the next two years are reported in Table 4. In order to test for a possible non-linear relationship, the occupation-specific probability of computerization enters the model together with its squared term. The results suggest that there is a statistically significant reversed U-shaped relationship between the risk of computerization and the self-reported probability of losing current paid employment (Column I) as well as the likelihood of giving up working in one's current occupational field and starting a job in a completely new occupation (Column II). This result is quite surprising because it means that the risk of occupational change increases with the rising risk of automation only to a certain threshold level. People in occupations with a very high risk of automation are less likely than people in occupations with a medium risk to expect occupational changes related to losing their job or starting a job in a completely new field within the next two years. One possible reason for this finding may be related to existing labor market regulations with regard to employees' protection against dismissal.9 Alternatively, employees in occupations with a high risk of automation may be over-optimistic about the future of their occupations and, thus, underestimate the risk of losing their jobs. Next, the labor market polarization implies that the highest risk of automation applies to middle-skill workers in routine jobs. Moreover, these workers will be prone to downward occupational mobility when they are displaced by machines, unless they possess or acquire skills that are not susceptible to computerization, such as creativity or social intelligence. Hence, it can be expected that the higher the risk of automation of jobs, the higher is the likelihood of being demoted at one's current place of employment. The results in Column IV of Table 4 support this hypothesis: there is a statistically significant (although at a 10% level) effect of occupational automation risk on the probability of demotion at one's current place of employment within the next two years.10

Table 3. Probability of occupational changes in the next 2 years by the level of occupation-specific automation risk

Questions on the probability of occupational changes Risk of automation

Low (<30%) Medium (30-70%) (>7$%)

Will you lose your job? 17.39 22.05 23.03

Will you stop working in your current field and start working in a different one? 8.91 12.10 14.63

Will you be demoted at your current place of employment? 45.88 45.81 47.31

Will you attend courses or seminars to obtain additional training or qualifications? 50.89 37.85 27.90

Will you receive a salary or wage increase beyond the collectively negotiated wage increases? 16.72 17.10 13.73

Will you start working on a self-employed and/or freelance basis? 10.18 7.35 6.57

Source: author's calculations.

9 For instance, in Germany a much stricter dismissal protection legislation applies to employees at large firms more often than those at small firms [Bauernschuster, 2013]. If high-risk routine jobs are more likely to appear at large firms, then the people employed in such jobs may still be in a rather secure position, despite the high risk of automation of said jobs.

10 Model III does not provide evidence of a possible non-linear relationship.

Table 4. Risk of automation of jobs and the probability of occupational changes within the next 2 years

Parameters Self-reported (in t) probability of occupational changes within the next 2 years

I II III IV V VI VII

Occupation-specific probability of automation in t 0.245*** (0.082) 0.462*** (0.109) 0.041 (0.034) 0.019* (0.010) -0.889*** (0.092) -0.220** (0.104) -0.445*** (0.128)

Occupation-specific probability of automation in t, squared -0.211*** (0.082) -0.288*** (0.107) -0.024 (0.034) - 0.553*** (0.093) 0.059 (0.105) 0.163 (0.131)

Years of formal education 0.149*** (0.025) 0.171*** (0.032) 0.006 (0.009) 0.006 (0.009) 0.314*** (0.028) 0.100*** (0.031) 0.168*** (0.041)

Years of formal education, squared -0.006*** (0.001) -0.006*** (0.001) 0.000 (0.000) 0.000 (0.000) -0.009*** (0.001) -0.002* (0.001) -0.005*** (0.001)

Tenure -0.032*** (0.002) -0.036*** (0.003) 0.004*** (0.001) 0.004*** (0.001) -0.003 (0.002) -0.014*** (0.003) -0.034*** (0.003)

Tenure, squared 0.001*** (0.000) 0.001*** (0.000) -0.000*** (0.000) -0.000*** (0.000) 0.000 (0.000) 0.000** (0.000) 0.000*** (0.000)

Experience of unemployment, years 0.028*** (0.004) 0.027*** (0.005) 0.004** (0.002) 0.004** (0.002) -0.019*** (0.005) -0.030*** (0.006) 0.001 (0.007)

Age 0.022*** (0.004) 0.034*** (0.006) 0.016*** (0.001) 0.016*** (0.001) 0.024*** (0.004) 0.039*** (0.005) 0.047*** (0.007)

Age, squared -0.000*** (0.000) -0.001*** (0.000) -0.000*** (0.000) -0.000*** (0.000) -0.001*** (0.000) -0.001*** (0.000) -0.001*** (0.000)

Gender (=1 if male, = 0 if female) -0.039*** (0.014) -0.027 (0.018) -0.032*** (0.006) -0.032*** (0.006) 0.078*** (0.016) 0.144*** (0.018) 0.178*** (0.023)

Nationality (=1 if German, = 0 if otherwise) 0.108*** (0.026) 0.070** (0.035) 0.007 (0.011) 0.007 (0.011) 0.247*** (0.032) 0.035 (0.033) 0.007 (0.045)

Presence of children in the household -0.091*** (0.013) -0.081*** (0.017) -0.003 (0.005) -0.003 (0.005) -0.049*** (0.014) -0.064*** (0.016) 0.007 (0.021)

Willingness to take risks 0.001 (0.003) 0.017*** (0.004) -0.006*** (0.001) -0.006*** (0.001) 0.020*** (0.003) 0.031*** (0.004) 0.042*** (0.005)

Conscientiousness -0.044*** (0.007) -0.082*** (0.009) -0.013*** (0.003) -0.013*** (0.003) -0.006 (0.008) -0.004 (0.009) -0.083*** (0.012)

Extraversion -0.021*** (0.006) -0.002 (0.007) -0.014*** (0.002) -0.014*** (0.002) 0.011* (0.007) 0.024*** (0.007) -0.013 (0.010)

Agreeableness -0.013** (0.007) -0.027*** (0.008) 0.001 (0.003) 0.001 (0.003) -0.008 (0.007) -0.036*** (0.008) -0.046*** (0.011)

Openness to experience -0.017*** (0.006) 0.043*** (0.007) -0.006** (0.002) -0.006** (0.002) 0.070*** (0.006) 0.022*** (0.007) 0.113*** (0.010)

Neuroticism (reversed emotional stability) 0.070*** (0.005) 0.069*** (0.007) 0.024*** (0.002) 0.024*** (0.002) 0.004 (0.006) -0.030*** (0.007) 0.025*** (0.009)

Internal locus of control -0.054*** (0.007) -0.023*** (0.009) -0.009*** (0.003) -0.009*** (0.003) 0.008 (0.008) 0.012 (0.009) -0.003 (0.012)

External locus of control 0.062*** (0.007) 0.053*** (0.009) 0.012*** (0.003) 0.012*** (0.003) -0.041*** (0.008) -0.005 (0.010) -0.037*** (0.013)

Year fixed effects Yes*** Yes*** Yes*** Yes*** Yes*** Yes*** Yes***

Industry fixed effects Yes*** Yes*** Yes*** Yes*** Yes*** Yes*** Yes***

Constant -1.617*** -0.203 -2.712*** -0.263 -0.480*** -0.078 -0.477*** -0.078 -3.066*** -0.227 -2.390*** -0.253 -3.098*** -0.346

Number of observations 30 890 30 901 30 677 30 677 30 885 30 717 29 600

Log Likelihood -12.201 -8.469 -14.314 -14.314 -15.190 -10.293 -6.821

Dependent variable: self-reported probability of occupational changes within the next two years. Robust standard errors in parentheses. Questions on the probability of occupational changes: I — Will you lose your job?; II — Will you stop working in your current field and start working in a different one?; III — Will you be demoted at your current place of employment?; IV — Will you attend courses or seminars to obtain additional training or qualifications?; V — Will you receive a salary or wage increase beyond the collectively negotiated wage increases?; VI — Will you start working on a self-employed and/or freelance basis? *** p<0.01, ** p<0.05, * p<0.1. Source: author's calculations.

At the same time, people in occupations with the highest and the lowest risks of automation are significantly more likely than people in jobs with a medium risk of automation to expect to gain additional qualifications in the near future (Column V). This observation may be indicative of a moderate risk of downward occupational mobility in the future, as people who urgently require additional skills in order to protect themselves from the negative effects of automation are likely to do so.11 Similarly, gaining additional qualifications may be also of high importance for people in occupations with a low risk of automation, in which computers strongly complement human labor. Moreover, a higher risk of automation is less likely to be associated with the probability of wage increases for individuals in those occupations (Column VI). Finally, an interesting result was obtained with regard to the probability of setting up one's own business, namely, the respondents in occupations with a low risk of automation are significantly more likely than people in jobs with a high risk of automation to see themselves moving toward self-employment in the near future (Column VII). This is an important result that points toward opportunity-driven nascent entrepreneurship, since the willingness to set up one's own business is more pronounced among workers who are in relatively secure jobs in terms of their susceptibility to automation. Moreover, jobs with a low risk of automation include tasks requiring creativity, social interactions and abstract thinking, which are critically important for entrepreneurs.

As regards the effects of the control variables, it was observed that people with high and low levels of education are significantly less likely to expect occupational changes, as compared to individuals with middling levels of education. People with a longer tenure, males as well as those with children, have a lower probability of occupational changes. With regard to the effects of personality traits, people with higher levels of conscientiousness, agreeableness and an internal locus of control are less likely to expect occupational changes in the near future. The results for other personality traits are mixed. For instance, people with greater willingness to take risks report a higher probability of starting a job in a new field, obtaining additional qualifications, expecting a wage increase, and of becoming self-employed.12 On contrary, less risk-averse people report a lower probability of demotion at their current place of employment.

Transitions to self-employment and unemployment

While the previous section presented the results for expected occupational changes, in this section the analysis is focused on real occupational transitions from paid employment to unemployment and self-employment within the next two years. The first column of Table 5 suggests that the high risk of automation in an occupation is more likely to lead to unemployment in the next two years. This relationship is linear, meaning that there is no decrease in the probability of unemployment for those workers who are in jobs with the highest risk of automation.13 Hence, it is likely that those workers underestimate the risk of losing their job, as has been shown in the Model I of Table 4.

Moreover, there is a statistically significant and negative relationship between the risk of automation and the transition from paid employment to self-employment. In order to provide a more differentiated picture of the relationship between the risk of automation and switches to self-employment, we additionally distinguish between self-employment with and without employees. The group of the solo self-employed may contain necessity-driven entrepreneurs, that is, those individuals who were at high risk of losing their jobs due to, for example, automation. On the contrary, the group of self-employed with employees is likely to contain opportunity-driven and growth-oriented entrepreneurs. Hence, one can expect different effects from automation risk on the various types of entrepreneurship. However, the results in Columns III and IV of Table 5 suggest that both types of entrepreneurs are likely to come from low-risk occupations, although the effect is lower (and only statistically significant at the 10% level) for the solo self-employed.

Conclusions

New and emerging technologies will lead to radical transformations on labor markets. As machines learn to accomplish not only routine tasks but also activities that require abstract skills and the ability to work in an unstructured environment, concerns arise that automation will wipe out a great number of jobs. This paper provided new evidence on the impact of the automation of jobs on individual-level occupational mobility. In particular, it shows that the expected probability of occupational changes rises with the occupation-specific risk of automation. This is particularly pronounced for such occupational changes as losing a job, demotion at one's current place of employment, or starting a new job in a different field. According to the respondents' own assessments, these changes are likely to occur within the next two years. This is quite in line with the prediction by Frey and Osborne [Frey, Osborne, 2013, 2017], who concluded that the current state of technology is such that it will be possible in the next 10-20 years to replace about half of the jobs in the US with machines.

11 Unfortunately, the data does not provide information with regard to skills the respondents are going to improve or attain by means of attending additional training.

12 This is in line with previous studies (see, e.g., [Caliendo et. al., 2009]).

13 A test of non-linearity proved to be insignificant in all model specifications of Table 5.

Table 5. Risk of the automation of jobs and transitions to self-employment and unemployment

Parameters Employment status in (t+2)

Unemployed Self-employed Self-employed without employees Self-employed with employees

I II III IV

Occupation-specific probability of automation in t 0.182*** (0.061) -0.244*** (0.079) -0.177* (0.092) -0.283** (0.111)

Years of formal education -0.130** (0.053) 0.12 (0.085) 0.185* (0.104) 0.016 (0.108)

Years of formal education, squared 0.004* (0.002) -0.002 (0.003) -0.005 (0.004) 0.001 (0.004)

Tenure -0.062*** (0.005) -0.034*** (0.007) -0.048*** (0.008) -0.009 (0.009)

Tenure, squared 0.001*** (0.000) 0.001*** (0.000) 0.001*** (0.000) 0.000 (0.000)

Experience of unemployment, years 0.095*** (0.008) -0.01 (0.014) -0.002 (0.016) -0.028 (0.025)

Age -0.036*** (0.008) -0.001 (0.013) -0.008 (0.015) 0.01 (0.019)

Age, squared 0.001*** (0.000) 0.000 (0.000) 0.000 (0.000) 0.000 (0.000)

Gender (=1 if male, = 0 if female) 0.034 (0.035) 0.131*** (0.047) 0.087 (0.054) 0.176** (0.070)

Nationality (=1 if German, = 0 if otherwise) -0.043 (0.058) -0.107 (0.088) -0.032 (0.117) -0.176 (0.110)

Presence of children in the household -0.058* (0.033) 0.132*** (0.045) 0.125** (0.053) 0.107* (0.063)

Willingness to take risks 0.006 (0.007) 0.058*** (0.010) 0.050*** (0.011) 0.059*** (0.015)

Conscientiousness -0.004 (0.019) 0.014 (0.027) -0.006 (0.031) 0.042 (0.038)

Extraversion -0.019 (0.014) 0.017 (0.020) -0.001 (0.022) 0.039 (0.029)

Agreeableness -0.017 (0.016) -0.039* (0.023) -0.04 (0.025) -0.032 (0.034)

Openness to experience 0.014 (0.014) 0.088*** (0.022) 0.107*** (0.025) 0.041 (0.029)

Neuroticism (reversed emotional stability) 0.028** (0.013) 0.022 (0.018) 0.042** (0.021) -0.013 (0.024)

Internal locus of control 0.012 (0.016) 0.032 (0.022) 0.023 (0.025) 0.036 (0.033)

External locus of control 0.093*** (0.017) -0.008 (0.024) -0.013 (0.029) 0.002 (0.032)

Year fixed effects Yes*** Yes *** Yes *** Yes ***

Industry fixed effects Yes *** Yes *** Yes *** Yes ***

Constant -0.23 (0.418) -4.279*** (0.686) -4.443*** (0.811) -3.927*** (0.850)

iНе можете найти то, что вам нужно? Попробуйте сервис подбора литературы.

Number of observations 47 123 47 048 47 048 46 553

Log Likelihood -5.219 -2.796 -1.816 -1.323

Notes: Dependent variable: Employment status in (t+2). Results of probit regressions. Robust standard errors in parentheses. *** p<0.01, ** p<0.05, * p<0.1. Source: author's calculations.

An important question then concerns which additional skills workers in jobs at high risk of automation should acquire in order to make themselves less susceptible to the negative consequences of automation. The empirical results in this paper provide evidence that workers in high-risk occupations do indeed intend to gain additional qualifications and training in the near future. However, no information was available with regard to the type of training they were more likely to choose. Hence, more research is needed in order to develop educational strategies to make workers less susceptible to automation.

Moreover, this paper investigated the relationship between the automation of jobs and an individual's propensity to become an entrepreneur. The causes for the recent rise of entrepreneurial activities in many innovation-driven economies including Germany, and in particular, the rise of self-employment without employees, are still not entirely clear. There is also concern regarding the quality of such businesses without

employees. For instance, a high risk of automation for certain jobs may drive entrepreneurial activities out of necessity. However, the result of this paper is that people in occupations at high risk of automation are significantly less likely than people in low-risk occupations to become nascent entrepreneurs and make the transition from paid employment to self-employment. This result holds for both self-employed with and without employees. It indicates that new technologies are likely to create new entrepreneurial opportunities in occupations that consist of tasks that are less likely to be computerized in the near future. Thus, the rising level of entrepreneurial activities is more likely to be driven by new technologies creating many new opportunities rather than by technology making jobs obsolete over the course of automation.

References

Acs Z.J. (2011) High-impact Firms: Gazelles Revisited. Elgar Handbook of Research on Entrepreneurship and Regional

Development — National and Regional Perspectives (ed. M. Fritsch), Cheltenham: Edward Elgar, pp. 133-174. Arntz M., Gregory T., Zierahn U. (2016) The Risk of Automation for Jobs in OECD Countries: A Comparative Analysis (OECD Social, Employment and Migration Working Paper no 189), Paris: OECD. DOI: http://dx.doi.org/10.1787/5jlz9h56dvq7-en.

Astebro T., Thompson P. (2011) Entrepreneurs, Jacks of All Trades or Hobos? Research Policy, vol. 40, pp. 637-649. Audretsch D.B., Thurik A.R. (2000) Capitalism and democracy in the 21st century: From the managed to the

entrepreneurial economy. Journal of Evolutionary Economics, vol. 10, pp. 17-34. Audretsch D.B., Thurik A.R. (2001) What is new about the new economy: Sources of growth in the managed and

entrepreneurial economies. Industrial and Corporate Change, vol. 10, pp. 267-315. Autor D.H. (2015) Why Are There Still So Many Jobs? The History and Future of Work-place Automation. Journal of

Economic Perspectives, vol. 29, no 3, pp. 3-30. Autor D.H. (2014) Polanyi's Paradox and the Shape of Employment Growth (NBER Working Paper no 20485), Cambridge,

MA: National Bureau of Economic Research. Autor D.H., Dorn D. (2013) The Growth of Low-Skill Service Jobs and the Polarization of the US Labor Market.

American Economic Review, vol. 103, no 5, pp. 1553-1597. Autor D.H., Levy F., Murnane R.J. (2003) The Skill Content of Recent Technological Change: An Empirical Exploration.

Quarterly Journal of Economics, vol. 118, no 4, pp. 1279-1333. Bauernschuster S. (2013) Dismissal protection and small firms' hirings: Evidence from a policy reform. Small Business

Economics, vol. 40, no 2, pp. 293-307. Baumol W.J. (2004) Entrepreneurial enterprises, large established firms and other components of the free-market

growth-machine. Small Business Economics, vol. 23, pp. 9-21. Brynjolfsson E., McAfee A. (2014) Race Against the Machine: How the Digital Revolution is Accelerating Innovation, Driving Productivity, and Irreversibly Transforming Employment and the Economy, New York: W. W. Norton & Company.

Caliendo M., Fossen F.M., Kritikos A.S. (2009) Risk attitudes of nascent entrepreneurs — New evidence from an

experimentally-validated survey. Small Business Economics, vol. 32, no 2, pp. 153-167. Caliendo M., Kritikos A.S. (2010) Start-Ups by the Unemployed: Characteristics, Survival and Direct Employment

Effects. Small Business Economics, vol. 35, no 1, pp. 71-92. Chang J.-H., Huynh P. (2016) ASEAN in Transformation: The Future of Jobs at Risk of Automation. Bureau for

Employers' Activities (Working Paper no 9), Geneve: Internaltional Labour Office. Costa P.T., McCrae R.R. (1992) Revised NEO personality inventory (NEO-PI-R) and NEO five factor inventory (NEO-

FFI) professional manual, Odessa, FL: PAR. Dengler K., Matthes B. (2015) Folgen der Digitalisierung für die Arbeitswelt. Substitu-ierbarkeitspotentiale von Berufen in

Deutschland (IAB Forschungsbericht 11/2015), Nuremberg: Institute for Employment Research. Frey C.B., Osborne M.A. (2013) The future of employment: How susceptible are jobs to computerisation?, Oxford, UK: University of Oxford.

Frey C.B., Osborne M.A. (2017) The future of employment: How susceptible are jobs to computerisation?, Technological

Forecasting and Social Change, vol. 114, pp. 254-280. Fritsch M. (2011) The Effect of New Business Formation on Regioonal Development: empirical evidence, interpretation, and avenues for further research. Elgar Handbook of Research on Entrepreneurship and Regional Development — National and Regional Perspectives (ed. M. Fritsch), Cheltenham: Edward Elgar, pp. 133-174. Fritsch M., Kritikos A.S., Rusakova A. (2012) Who Starts a Business and Who is Self-Employed in Germany? (Jena

Economic Research Paper 001-2012), Jena: Friedrich Schiller University; Max Planck Institute of Economics. Fritsch M., Kritikos A.S., Sorgner A. (2015) Why did self-employment increase so strongly in Germany? Entrepreneurship

and Regional Development, vol. 27, pp. 307-333. Gerlitz J.Y., Schupp J. (2005) Zur Erhebung der Big-Five-basierten Persönlichkeits-merkmale im SOEP (Research Notes),

Berlin: German Institute for Economic Research. Goos M., Manning A., Salomons A. (2014) Explaining Job Polarization: Routine-Biased Technological Change and

Offshoring. American Economic Review, vol. 104, no 8, pp. 2509-2526. Pajarinen M., Rouvinen P., Ekeland A. (2015) Computerization Threatens One-Third of Finnish and Norwegian

Employment (ETLA Brief no 34), Helsinki: Research Institute of the Finnish Economy. Papke L.E., Wooldridge J.M. (1996) Econometric Methods for Fractional Response Variables with an Application to

401(k) Plan Participation Rates. Journal of Applied Econometrics, vol. 11, no 6, pp. 619-632. Parker S.C. (2009) The Economics of Entrepreneurship, Cambridge: Cambridge University Press. Shane S. (2000) Prior Knowledge and the Discovery of Entrepreneurial Opportunities. Organization Science, vol. 11, no 4, pp. 448-469.

Shane S. (2009) Why encouraging more people to become entrepreneurs is bad public policy. Small Business Economics, vol. 33, pp. 141-149.

Sorgner A., Bode E., Krieger-Boden C. (2017a) The Effects of Digitalization on Gender Equality in the G20 economies.

Kiel: Kiel Institute for the World Economy. Sorgner A., Fritsch M., Kritikos A.S. (2017b) Do Entrepreneurs Really Earn Less? Small Business Economics, 2017,

vol. 49, no 2, pp. 251-272. DOI 10.1007/s11187-017-9874-6. Wagner, G.G., J.R. Frick, and J. Schupp (2007) The German Socio-Economic Panel Study (SOEP) — Scope, Evolution

and Enhancements. Schmollers Jahrbuch, vol. 127, no 1, pp. 139-169. Welter F. (2006) Women's entrepreneurship in Germany: Progress in a still traditional environment. Growth-oriented Women Entrepreneurs and Their Businesses: A Global Research Perspective (eds. C.G. Brush, N.M. Carter, E.J. Gatewood, P.G. Greene, M.M. Hart), Cheltenham: Edward Elgar, pp. 128-153.

i Надоели баннеры? Вы всегда можете отключить рекламу.