CC BY-NC-ND
38
Original article
IMPLEMENTATION OF MONITORING OF THE QUALITY OF PUBLIC SERVICES AT THE LOCAL LEVEL AS A TOOL FOR REFORMING PUBLIC ADMINISTRATION -THE POLISH EXPERIENCE
Tomasz Michalski1, Krzysztof Kopec2
1 PhD, Associate Professor at the University of Gdansk Institute of Socio-Economic Geography and Spatial Management, Department of Regional Development. Address: Jana Bazynskiego 4, 80-309 Gdansk, Poland
E-mail: tomasz.michalski@ug.edu.pl
2 PhD, Assistant Professor at the University of Gdansk Institute of Socio-Economic Geography and Spatial Management, Department of Regional Development. Address: Jana Bazynskiego 4, 80-309 Gdansk, Poland
E-mail: krzysztof.kopec@ug.edu.pl
Abstract. The main aim of the article is to present a model of monitoring of the quality of public services which can be used to enhance the quality of self-government in European post-communist countries. The secondary aim of the article is to identify factors which hinder the implementation of the model. The theoretical basis for the presented model comes from three earlier ones: the Deming cycle, the Boyd cycle, and the model of institutional development of organizations.
The model has been tested in 10 local administration units (LAU) in Poland. It consists of six stages presented in the article: decision to implement monitoring, its objectives and resources; selection of indicators; collection of data; analysis of the collected data and preparation of reports; use/dissemination of the reports; final evaluation of monitoring and implementation of conclusions. Three critical points were identified at which decisions to stop monitoring were often taken, namely: disappointment with the first main report which failed to answer all of the problems raised by the authorities, changes of the LAU authorities as a result of elections, and "indolence" of local authorities manifested in expecting support from outside.
Keywords: model of monitoring, quality of public services, local level of public administration, reforming public administration, Poland.
128
© HSE, 2021
For citation: Michalski, T. and Kopec K. (2021) 'Implementation of monitoring of the quality of public services at the local level as a tool for reforming public administration - the Polish experience', Public Administration Issues, 6 (Special Issue II, electronic edition), pp. 128-145 (in English). DOI: 10.17323/1999-5431-2021-0-6-128-145.
JEL Classification: H41, H75.
Introduction
There are many theories related to defining public services. This study uses the concept of public interest, which can also be seen as a starting point for defining public administration. The notion of public interest rather escapes unambiguous definitions - it requires constant redefinitions resulting from variability in its interpretation. Consequently, its different definitions depend on the public management model (Denhardt and Denhardt, 2007; Rosenbloom, 1993):
1. In the Public Administration model, public interest serves higher aims and represents citizens' interests, but the power to define it and include it in legal acts lies in democratically elected politicians.
2. In the New Public Management model, public interest is viewed as an aggregated interest of members of the society reflected in the interests of the winning coalition.
3. In the good governance model, public interest is a goal set as a result of participation of citizens and entities created by them in the process of governance.
If the concept of public interest is adopted as a starting point to define public services, then their definition is formulated in the process of social construction. This means that a certain society at a given time is an active creator of this definition. As a result, society as a whole and through its representatives decides on what public services are, what goods they include, what their availability and uncompetitive use is (Nemec, Murray Svidroñová and Kovács, 2019), and whether they should be delivered to all citizens on a similar basis or with certain preferences to special groups - e.g. the poor or the disabled.
There is a multitude of definitions of monitoring (Public Service Commission South Africa, 2008; Czochanski, 2013; Styrin and Plaksin, 2012), but most of them have the following in common: (1) regularity of data collection according to strictly specified rules and (2) use of data to find out the extent of achieving the objectives or using the allocated funds. As regards monitoring of the functioning of public institutions, there is an additional aspect of improving their functioning (Dascálu, Marcu and Hurjui, 2016; Yuzhakov, 2014). Therefore, monitoring of the quality of public services can also be treated as an element of management.
After 1989, a whole series of transformation processes took place in European post-Communist countries whose aim was to improve, at least theoretically, the quality of their citizens' lives (see Norkus, 2012; Rose, 2009). A reform of administration is one of such processes.
A whole spectrum of activities is undertaken at the operational level as part of the reform of public administration in European post-communist countries. On the one hand, they are fundamental in nature and include such processes as:
- changes in the administrative division (Vanags and Vilka, 2006; Meyer-Sah-ling and Yesilkagit, 2011);
- decentralization (Davey and Peteri, 2006; Maksimovska, Stojkov and Schmidt, 2015; Tikhomirov, 2013);
- division into "optimal" municipalities (Campbell and Coulson, 2006; Kucz-abski, Zastavetska and Zastavetskyy, 2017).
On the other hand, there are a number of actions aimed at changing the attitudes of local government officials or at implementing various innovations to their work. This study belongs to the second type of actions.
However, despite a wide range of reforms, many researchers assume that public administration in Eastern and Central-Eastern Europe is inferior to the West-European version (see: Bouckaert, 2009; Neshkova and Kostadinova, 2012).
In this context, the main aim of this study is to build a model of monitoring of the quality of public services (in accordance with principles of both New Public Management and good governance) which could be implemented by Polish local governments. The secondary aim is to trace how this model is received by the Polish local authorities.
Prior to the study, a hypothesis was formulated that although developing and then implementing such a model is possible in Poland, its application may be short-lived. This process requires involvement of local government authorities and officials and it is exposed to threats that may lead to abandoning the monitoring. This may happen even if such monitoring brings important information about changes in the quality of public services.
Research procedure and the applied methods
The first stage of the research procedure was to develop a model for monitoring the quality of public services. This was done in three phases. First, our own original version of the model was developed based on other models. In the second phase, it was tested (in 2012-2014) in local governments. During this phase, the model was subject to some changes and alterations. The third phase covered the years 2015-2019 during which the working model was tested in local governments. Unfortunately, the COVID-19 pandemic meant that its testing was discontinued in 2020. Resuming the research is planned after the end of the pandemic.
During the research, an inductive procedure was followed, namely the aim was to achieve a universal but also exemplary approach to the issue. This meant generalization and elimination of distorting factors (Fielding, 1988; Silverman, 2014). The nomothetic approach was applied. As a result, elements linking cases in individual local administration units were searched for (Gibbs, 2007; Lancaster, 2005).
Three other models were used in the construction of this model. The first two (the Deming cycle and Boyd's cycle) are of a general nature and de facto are cycles
of observation, decision-making and action. Because of their similarity, they can be considered as complementary to each other.
In the Deming cycle1, four successive groups of actions are distinguished based on the sequentially of actions undertaken in the following pattern: Plan-Do-Study-Act (Deming, 1993):
1. Plan - identify the needs, objectives and scope of monitoring, and design the system.
2. Do - collect data and calculate indicators, and then report them.
3. Study - control the system in terms of its efficiency and effectiveness and its compliance with the proposed objectives.
4. Act - maintain the functioning of the system if it is found to operate properly or implement corrective actions, measures, etc., based on the information obtained from verification procedures.
In the case of Boyd's cycle (also called the OODA cycle), similarly to the previous model, four groups of actions are distinguished in the following pattern: Observe-Orient-Decide-Act. However, what is very important in this model is that particular groups of actions can be implemented simultaneously (Boyd, 2010).
The third model was developed in Poland (this is the so-called model of institutional development of organizations). It was developed as part of the Institution Development Program carried out in Poland in 2002-2004 by a consortium of the Canadian Urban Institute and the Malopolska School of Public Administration of the Cracow University of Economics. The main elements include assessment of the status of the organization, design of institutional changes (improvements), their implementation, and analysis and correction of the process (Bober et al., 2004).
To achieve the secondary aim, a publication by Struyk (2007) was used. Based on an analysis of the implementation of 18 demonstration projects in Russia during the transition, the author distinguished nine factors of successful implementation of a program: a degree of local leadership support and its consistency, characteristics of the implemented policy, availability of resources, the number of implementing actors, the implementing personnel's attitude, alignment of clients, the opportunity for learning among implementers, past experience, and the local environment.
The study was conducted in 10 LAUs in Poland: in city counties of Gdansk and Slupsk, urban municipalities of Chojnice and Puck, urban-rural municipalities Czarna Woda and Siedliszcze and rural municipalities of Stegna, Morzeszc-zyn, Puck and Melgiew.
The basic unit of local administration (LAU) in Poland is the municipality (LAU-2). There are three types of municipalities: rural, urban and rural-urban. Several municipalities form a county (LAU-1). A city county (a city with the
1 Deming's cycle has many names, such as the PDCA cycle (Plan-Do-Check-Act) or the PDSA cycle (Plan-Do-Study-Act). It is also found in the literature under the name of the Shewhart cycle. In reality, however, the Shewhart cycle looks slightly different - it consists of three stages: specification, production and inspection (Shewhart, 1937). Only later did Deming slightly modify this cycle by adding the fourth stage.
rights of a county) is a specific type of municipality. This status is held by cities with a population of over 100,000 inhabitants and cities with important functions in the settlement structure. They simultaneously perform tasks assigned to both municipalities and counties.
Description of the model of monitoring
The previously mentioned models formed a basis for developing a model of monitoring the quality of public services at the local level. On their basis, the first conceptual model (Michalski, 2014) was developed, which underwent a number of changes resulting from the experience gained during cooperation with local government officials. The version presented in Figure 1 can be considered final. The proposed model consists of six stages, which are characterized in detail in the further sections of this study.
Taking a decision on the implementation of monitoring and its objectives and allocated resources
Selecting indicators for monitoring
Collecting data based on selected indicators
Analyzing the collected data and preparing reports
Using/disseminating the reports
_±_
Final evaluation of monitoring and implementation of conclusions
Figure 1: Model of monitoring the quality of public services.
Stage one: Taking a decision on the implementation of monitoring and its objectives and allocated resources
The first stage consists of two parts. The first one is political in nature and is associated with taking (or not) a decision to implement monitoring of the quality of public services in a given LAU. Previous experience in the implementation of monitoring shows that in the vast majority of cases this decision is taken individually by the head of the local executive authority or by two persons: by the aforementioned executive after consulting the president of the local legislative authority.
After a positive decision has been taken by the authorities, in the second part, objectives and tasks are formulated, and resources that can be used and means that can be allocated for its implementation are reviewed. As practice
shows, while authorities are eager to engage in the discussion on objectives and tasks of monitoring, the review of resources and means is entirely assigned to officials. At this stage, two threats can be diagnosed. The first one is related to the objectives and tasks of monitoring which are too ambitious for the allocated means and resources. Ex ante evaluation is proposed as a solution. The second risk is related to the fact that some heads of departments in municipal offices believed that monitoring would be mainly used to assess the work of their departments. The applied solution of explaining the objectives of monitoring was only partially successful. About 40% of department heads having reservations could not be persuaded.
Stage two: Selecting indicators for monitoring
At this stage, there are also two parts. The first one is to choose indicators. Previous observations related to the implementation of this model show that at this stage of work it is very important to appoint a person responsible for the efficient realization of monitoring as early as possible and to give him/her the necessary powers by the local authorities. The research shows that if such a person has too little power, the indicators are not correctly selected. They are not chosen based on the defined objective of monitoring but are a result of a "game" of department managers.
Based on several years' experience in working with local governments, a four-level system of selecting indicators (Figure 2) later used in monitoring was developed. The previous procedures of selecting indicators very often led to: (1) "getting lost" of the appointed officials while selecting indicators, which often resulted in insufficient holism while diagnosing the condition of public services; (2) encouraging the "exclusion" of whole tasks from monitoring due to the behind-the-scenes activities of some department managers who were afraid that the effects of their work would be monitored.
Figure 2: Stages of selecting indicators.
The first two stages are preliminary (Table 1). In their framework, five areas of activity have been distinguished (the first level of selection) to which tasks are assigned (the second level of selection). Only after completing the second level is the selection of specific indicators made - their number varies from one to seven per specific task.
Table 1
The two first stages of selecting indicators: areas of activity and tasks
Area of activity: Task:
Administration Office availability/accessibility for clients
Compliance of administrative decisions with the law
Providing services for entrepreneurs by the office
Providing services for residents by the office
Condition of spatial order
Financial policy of the local administration
Online communication with customers
Assistance and welfare Health prophylaxis among residents
Conducting health policy (without prophylaxis)
Premises and places for the homeless and the poorest
Assistance for the poorest and the disabled
Safety and order
Public utilities Access to the water supply network
Access to the sewage network and wastewater treatment
Atmospheric air condition
Noise level
Maintaining cleanliness in the municipality
Condition of transport ways
Condition of local public transport
Culture and leisure Cultural/entertainment offer of local authorities
Conditions for outdoor leisure
Sports offer of local authorities
Education Level of exam results
Level of computerization in schools
Safety at school
Pre-school care
Pupils and parents-friendly school
Source: Czerwinska et al. (2014).
In total, local authorities deal with four types of indicators. The first three types are already prepared in the form of indicator sheets. These are so-called main indicators (single and complex) and variant indicators. With this approach, a user chooses the main indicators (the third level of selection) and, if it is the so-called single main indicator, by using a ready measurement card, he/ she immediately has instructions on how to gather data to calculate the value of the indicator and its recommended interpretation. When he/she selects the so-called complex main indicator, then he/she must select a particular variant indicator (the fourth level of selection) and only then get instructions on how to obtain the data to calculate the value of the indicator and its recommended interpretation.
The complex main indicators and variant indicators related to them were primarily introduced in order not to "obscure" the image of the service with an excessive number of very detailed indicators. This solution is used in a number of cases, most often:
- when data on a specific public service can be obtained in a number of ways, e.g. data on punctuality of public transport: direct observation, results of measurements carried out by carrier/carriers (if any), surveys;
- when there is more than one type of institution providing a specific type of service, e.g. there are up to three types of schools for which local authorities are responsible: elementary school, junior high school and senior high school - hence there are three separate variant indicators, e.g. to find out parents' opinion on safety in the public school their children go to;
- when there is more than one type of provided services, e.g., data on the number of people in families benefiting from social welfare can be obtained in an aggregated form, but also one specific reason for getting this assistance can be chosen, e.g. poverty, homelessness, being an orphan, unemployment, long-term illness, domestic violence, helplessness in caring matters, etc.
The fourth type is specific indicators. Monitoring of the quality of public services is open to amendments and modifications. As a result, these indicators are developed independently by auditors monitoring the quality of public services in their LAUs. They have been introduced for three reasons: (1) to make the monitoring system as holistic as possible, (2) not to include such indicators in a basic database that could be useful only in a very small number of LAUs, and thus they would obscure the transparency of the whole tool, (3) to make the selection stage flexible, i.e. not to limit the local authorities' creativity. Such an indicator could be exemplified by the capacity of a sewage treatment plant in a situation of increased tourism (measured in BOD5 from 1 person staying in a given LAU equal to 60g O2 /day).
Stage three: Collecting data based on selected indicators
The data collected to calculate the values of indicators selected for monitoring come from two types of sources (Figure 3). The first type is based on desk research, while obtaining data from the other type of sources is much more costly since they require conducting surveys or direct measurements.
Figure 3: Data sources for indicators.
The data collected as part of desk research come from the Internet (the first two points) or are collected based on inquiries directly sent to different institutions (the remaining points):
1. Databases of Statistics Poland (GUS), mainly the Local Data Bank. This institution collects an increasing range of data useful for local analyses and available free of charge. However, the downside of gathering data in this way is a long delay in their publication, up to one year. Therefore, in some cases, retrieving the same data from local institutions is recommended (e.g., data on readership in municipalities).
2. Databases and reports available on the Internet and maintained by other institutions. For example, the Regional Examination Committees publish data on examinations conducted at the end of elementary school, junior high and senior high schools.
3. Data retrieved from inquiries sent to institutions not affiliated with the local administrative unit that conduct monitoring. Examples include the National Health Fund (NFZ) or regional boards of education.
4. Data retrieved from inquiries directly sent to the employees of a given LAU office or to an institution affiliated with it (e.g., a public library, a sewage treatment plant).
The next three sources of data indispensable to calculate the indicator values require direct research. In the democratic realities, the perception of the quality of public services seems to be as important as the actual level of their implementation. Therefore, survey studies are deemed necessary. Two such surveys were offered to local authorities.
The first one is intended for implementation among pupils of schools in which the local government is the founding authority. Since only a few indicators collected from these surveys are sufficient to prepare the main report on the quality of public services, and the costs of surveys are quite high, local authorities were advised to conduct more elaborate surveys (than it is necessary just
for the main report), and on their basis to prepare an additional report on the perception of the situation in schools (a similar suggestion was made in the case of the second survey).
Based on experience, it has been found that the process of conducting surveys must be strictly controlled, preferably by an external expert. Therefore, the influence of persons related to the school on the process of selecting the research group and the actual testing in the classroom has been minimized. Hence, stratified random sampling is recommended in choosing classes to be surveyed. In this method, choosing a school is the first stage, and a selection of a specific class within the chosen school is the second stage. While conducting a survey in the classroom, the teacher is asked to leave the room, so that only the interviewer and the pupils remain inside.
The second survey involves the adult population of the given local administrative unit. Initially, attempts were made to render the study results as statistically representative as possible to reflect the inhabitants' views. However, such highly representative surveys proved too expensive, so the authorities of particular LAUs did not want surveys in such a form. Therefore, the decision was made to conduct less accurate, but also much less expensive, surveys. For example, in rural municipalities, the number of surveys was proportionally divided among different villages and in urban municipalities among different districts. Naturally, the proportions regarding age and gender were ensured.
Sporadically, local authorities declared a need to develop specific indicators that required an additional survey. So far, this request was made twice - in both cases, it concerned conducting a survey among tourists.
In almost all local governments, conducting the survey in schools proved to be the biggest problem. Frequently, their principals feared that the survey results would be used to evaluate the school's performance, and hence they could be used to "attack" the school management and teachers working in a given school. Explanations that the results from a given school would only be known to the person conducting the survey and that the local government authorities would only receive aggregated results from all schools did not convince anyone. This shows a low level of social trust in Poland.
In rare cases, direct measurements were used as a source of data. However, due to enormous effort involved in acquiring this kind of data, local government representatives were very reluctant to this type of measurement. As a result, there were main reports lacking in even one indicator for which the data was obtained in this way.
Stage four: Analyzing the collected data and preparing reports
At the fourth stage, analysis is conducted based on the previously selected indicators. As mentioned above, because there is a need to conduct two surveys, it is suggested that additional questions should be added to the questionnaires, and then additional reports should be developed based on the results of these surveys.
The main report contains an analysis based on indicators. Preparation of two versions of this report is recommended. The basic version includes items 1-4 mentioned below. In order to be able to read the results quickly, its volume
should be within the range of 20 to 30 A4 pages. The expanded version includes all nine points, but items 5-9 should be available only on a CD attached to the paper version of the main report containing items 1 to 4.
1. An introduction explaining the objectives and tasks of monitoring the quality of public services and containing a concise characteristic of the data sources.
2. Analysis of the quality of the services, broken down into five areas of activity.
3. Conclusions and recommendations.
4. An appendix with a table containing the name of the indicator and its value (including the value of this indicator from previous studies is also recommended).
5. An appendix containing the sheets of all the indicators used in compiling the report.
6. An appendix containing the survey questionnaire for school pupils.
7. An appendix containing the survey questionnaire for the adult population.
8. An appendix containing a report from the survey conducted among pupils.
9. An appendix containing a report from the survey conducted among the adult population.
Both reports containing survey results should be in a typical form of this type of studies. The same regards reports from any additional surveys (as mentioned in the previous section).
Due to periods of increased workload of the officials, the months of April-May are considered the optimal time under Polish conditions to prepare the preliminary version of the report containing the data for the previous year. June-August should be dedicated to consultations, so that the final version of the report can be submitted by mid-September at the latest. This stage of work encountered the greatest problems in small LAUs. This was due to a small number of employees working in the municipal offices. As a result, not only was there a shortage of officials who could be delegated to prepare the report, but also the delegated persons did not have the appropriate qualifications to cope with this task. The authorities of these local governments expected that they would be constantly supported in monitoring by external experts, clearly declaring that in the absence of such support, they would resign from monitoring.
As regards the substantive aspect, the main difficulty in the first years of implementing the monitoring of the quality of public services lies in presenting the main report in a form useful for the local authorities (points 2 and 3 of the above list). It was considered appropriate not to compare the indicator values in the surveyed local administration unit with values for other units. Yet, in many cases, this is impossible. Although curiosity is a natural human trait - is it better or worse elsewhere? - the prepared main report should definitely not serve this purpose. Its primary aim is to provide reliable information on the quality of public services offered in the given LAU. In order to achieve this objective properly, data from at least 3-4 years are indispensable. This will enable local authorities (1) to have a reference point and (2) to be able to identify whether the quality of the services is changing.
The attitude of the authorities towards the usefulness of the results proved to be a major problem. Local authorities expect clear conclusions "here and now"
as to what the situation in terms of the quality of these services is like in their LAU. In addition, the electoral calendar has an influence (fears that since the full results of monitoring will be visible only in a few years' time, it may happen that someone else will be in power in this LAU). It is very difficult to minimize the risk associated with this expectation. Action has been taken to make the local authorities completely aware that they cannot expect immediate effects, that this is a long-term investment, etc. After these explanations, the authorities officially declared that they understood the situation. Nevertheless, according to unofficial information, they were frustrated with the lack of instant results that could be promptly used.
Moreover, in almost all the surveyed municipalities there was a desire to compare one's own results with the results obtained in other LAUs. The authorities wanted to know whether the quality of public services in the municipality they manage is better or worse than in other ones. However, the situation in their municipality for a year or two years should always be the point of reference for them. After all, the goal is to improve the quality of public services in that particular LAU, regardless of the quality of services in other municipalities.
Stage five: Using/disseminating the reports
The stage of using the results is extremely important. Based on several years' collaboration with local authorities, three likely scenarios of dissemination and implementation of the obtained results have been distinguished:
1. After preparing the report and having it read by a very narrow group of local administration officials (e.g., only the authorities of the given LAU), it is put on the proverbial "shelf". Therefore, the incurred expenses are wasted. Fortunately, during all projects so far, there was no such situation. But this seems to result from the fact that participation in monitoring was voluntary, and only those LAUs volunteered whose authorities were genuinely interested in implementation of monitoring of the quality of public services.
2. In the second scenario, there is typical dissemination; namely, the report is used on a small scale during the current policy-making in the given LAU. For example, it might be presented during the council session and discussed or used once during preparation of, for example, a new development strategy. In practice, nearly all of the local governments with which this project has been implemented follow this scenario.
3. The most desirable scenario for using the report is mainstreaming. In this scenario, the report results are incorporated into the mainstream policy of the local government, i.e., not only is it used repeatedly in different situations, but monitoring is also conducted in the following years. Unfortunately, only two LAUs followed this scenario. The past tense is used because the existing authorities had lost elections, and the new authorities were not interested in continuing the project (this is the second of the three critical points in implementation of the project - the change of authorities).
Focusing on the second and, in particular, the third of the above outlined scenarios of making use of the report, three practical ways of using it may be defined (Figure 4). They were developed based on observations in local governments.
Figure 4: Practical use of the results of monitoring the quality of public services.
Firstly, the report is used directly to improve citizens' quality of life. Here there is a whole range of possibilities whose limits are determined by two extreme options. In the first one, we diagnose the area of a high quality of life, and then we determine which public services are related to it and decide not to change anything in the services. In the second one, we diagnose the area of a clearly low quality of life, and then we determine which public services are connected with it and decide to take necessary action. There may be enough data to determine the quality of which public services should be improved. Yet, there may also be a case that additional, highly detailed monitoring with specially designed specific indicators should be conducted.
The second option is to use monitoring to improve the quality of the functioning of local administration and the related institutions (this is strongly linked to the ex post evaluation which is recommended in the final stage of the model).
The third possibility regards raising the level of public discourse in local communities. Observations show that in situations where all the debate participants have access to reports, the level of demagoguery and non-substantive statements is reduced. However, in this case the ultimate result strongly depends on the level of panelists' personal culture. Unfortunately, there were attempts to use the report results to fight the current local authorities. The critics of the authorities did not focus on how the public services in the municipality could be improved by using the report results. Instead, they used them to undermine the competences of the authorities and employees of the municipal office. This strongly fueled the officials' fears diagnosed in stage one.
Stage six: Final evaluation of monitoring and implementation of conclusions
The final stage is implemented when, after completing the entire measurement cycle, local authorities express their willingness to continue it. In this case, there is time for ex post evaluation and then implementation of its results in the next measurement cycle. Applying the typical methods used for this kind of evaluation is recommended. At this stage, the main risk to continuing monitoring is that the LAU authorities expect constant and free assistance from external experts in implementation of monitoring of the quality of public services. Such
a declaration was expressed by all the local governments that decided to pursue the project in the following years. Later experience of cooperation with local authorities showed that, after such free support/free assistance had ceased, almost all authorities discontinued monitoring the quality of public services.
Figure 5 presents the recommended procedure to evaluate monitoring of the quality of public services. As can be seen, it is followed in three stages, concerning the very subject of monitoring, then its tools and finally its products.
[1] SUBJECT OF MONITORING
Did implementation of monitoring provide information we wanted to have? (if not, what needs changing) Did we really monitor what we assumed in the beginning? (if not, what needs changing) Do we still want to monitor what have monitored so far? (justify why) Do we want to monitor something else? (why? what will it give us?)
> f
[2] TOOLS IN MONITORING
Is the grouping of indicators optimal? (e.g., does it facilitate the selection of indicators?) Are the indicators unequivocally developed? (e.g., are there no interpretative doubts? Are data sources appropriate (e.g., are they reliable? can data be retrieved from them)?
> f
[3] PRODUCTS OF MONITORING
Are we satisfied with the report form? (if not, how to change it?) Was the report disseminated at all? (if not, why?) Was the report used in accordance with the initially adopted assumptions (if not, why?) Did the applied paths of disseminating the report bring the desired effects? (if not, why?)
Figure 5: Suggested ex post evaluation procedure for the system of monitoring the quality of public services (Michalski, 2014).
The decision to continue monitoring was usually accompanied by substantial willingness to introduce changes in its implementation. This was partly due to the results of ex post evaluation and partly due to the natural willingness to make changes (identified as improvements). In general, this is a positive practice; however, as far as selecting indicators is concerned, the extent of changes should be limited because the system meets the requirements well by delivering valuable results - providing we have exactly the same data gathered by the same method data from at least four years (because only then can we trace the negative and positive changes in the quality of the provided services).
Conclusion
Nowadays, Polish local governments are increasingly interested in monitoring. This has resulted in a number of EU-funded or national programs as well as handbooks on all areas of local government activity (e.g., Czerwinska et al., 2014; Trutkowski (ed.), 2011). The proposed model of monitoring the quality of public services is part of this trend. Its advantage lies in first designing the theoretical concept, then testing it for three years, and then preparing the final product and implementing it in a few LAUs. This activity certainly contributes to raising the quality of functioning of self-government in Poland.
The hypothesis of the possibility of development and implementation of monitoring the quality of public services at the local level in Poland has been confirmed. Unfortunately, the fact that it can be used for a short time has also been confirmed. In particular, discontinuing external support was the moment when monitoring became limited or completely abandoned. This happened even when conducting it brought clear benefits in the form of information about changes in the quality of public services. At the general level, one may broadly state that this is due to a lower level of the culture of functioning of local administration than in the leading European Union countries. This completely supports the views of Dan and Pollitt (2015) as well as Drago§ and Neam^u (2007) that activities of a novel nature, in fact, improve the quality of functioning of local administration in Central and Eastern Europe, although this is happening with obstacles and at a slower pace than persons and institutions promoting these changes would wish.
As mentioned in the introduction, in order to identify the reasons for the frequent resignation from using this model by local governments (after discontinuing external experts' support), the method presented by Struyk (2007) was applied. On its basis, three critical points were identified at which decisions to stop monitoring were often made:
1. Disappointment with the first main report in which there were no answers to all of the questions raised by the LAU authorities. Very often they could not acknowledge that at least four years must pass (i.e., there must be reports from at least four years) to be able to fully appreciate and use the advantages of the proposed monitoring.
2. Changes of the authorities of the given LAU as a result of lost elections by the current authorities. At all levels, the new authorities' tendency to start everything from scratch is still too obvious.
3. "Indolence" of local authorities. As "indolence", we understand these authorities' expectations to be still supported in various ways with financial means from the EU. This could be direct support in the form of money transferred to a LAU for the realization of specific projects or intermediate in the form of support from experts very often remunerated from these funds. This led to situations in which, after such support was discontinued, even those LAU authorities who were satisfied with monitoring results resigned from it, explaining themselves by a lack of means and resources for its independent implementation. Looking at the above-mentioned threats (the three main ones above and a few
minor ones in the text), it can be concluded that the low level of social trust in insti-
tutions is a major threat to the implementation of monitoring of the quality of public services in Poland is (see: European Social Survey, 2018; Zaufanie w sferze..., 2006).
Seven conclusions can be drawn from the work being conducted in the ten municipalities, of which three are fundamental, and four are supplementary: (1) monitoring must not be an end in itself - it is only a tool for achieving other goals, (2) monitoring must address important issues; therefore, indicators must measure important things, (3) monitoring is discouraged by the amount of work necessary to conduct it (hence it should be limited) and the lack of instant effects, (4) monitoring must be resistant to tampering with the results, (5) no one should fear or be anxious about monitoring, (6) monitoring is not meant for comparisons between LAUs but to compare the current situation in a LAU with an earlier one -a year, two years earlier, etc., (7) the model of monitoring may be (and should be) supplemented and amended if the nature of a LAU, its problems or objectives justify such steps (Kopec, 2020). Action resulting from these conclusions should weaken or eliminate the indicated threats to the implementation and execution monitoring of the quality of public services. However, this could be a subject for the fourth stage of research, possible after the end of the COVID-19 pandemic.
Nevertheless, one may claim that in all LAUs in which monitoring of the quality of public services was implemented, including those where its implementation was eventually discontinued, it contributed to the reform of public administration. In every case, monitoring made the local authorities more open and willing to cooperate with different stakeholders - both external and internal, as well as with local residents. Thanks to this process, the participation of these stakeholders and residents in the exercise of power has increased. As a result, the level of accountability for the goals that were pursued has also increased. Finally, the participants in the monitoring process started cooperation with one another, forming a certain network. All this made public administration in a LAU which implemented monitoring of the quality of public services closer to the model of good governance.
REFERENCES
Bober, J., Mazur, S., Turowski, B. and Zawicki, M. (2004) Rozwoj instytucjonalny. Porad-nik dla samorzqdow terytorialnych [The institutional development. A guidebook for local governments]. Krakow: Malopolska Szkola Administracji Publicznej Akademii Ekonom-icznej w Krakowie and Ministerstwo Spraw Wewn^trznych i Administracji (in Polish).
Bouckaert, G. (2009) 'Public sector reform in Central and Eastern Europe', Halduskul-tuur, 10, pp. 94-104.
Boyd, J.R. (2010) The essence of winning and losing. Available at: http://pogoarchives.org/ m/dni/john_boyd_compendium/essence_of_winning_losing.pdf (accessed 21 May 2020).
Campbell, A. and Coulson, A. (2006) 'Into the mainstream: Local democracy in Central and Eastern Europe', Local Government Studies, 32(5), pp. 543-561.
Czerwiñska, M., Gajdasz, J., Hildebrandt, A., Kopec, K., Kupc-Muszyñska, B., Michalski, T., Nowicki, M., Susmarski, P. and Tarkowski, M. (2014) Jak zaprojektowac i wdrozyc system monitoringu jakosci uslugpublicznych i jakosci zycia? Podrçcznik wdrazania [How to design and implement a system of the monitoring of the quality of public services and he quality of life. A guidebook for implementation]. Gdañsk: IBnGR (in Polish).
Czochañski, J.T. (2013) Monitoring rozwoju regionalnego. Aspekty metodologiczne i imple-mentacyjne [Monitoring of the regional development. Methodological and implementation aspects]. Warszawa: KPZK PAN (in Polish).
Dan, S. and Pollitt Ch. (2015) 'NPM Can Work: An optimistic review of the impact of New Public Management reforms in central and eastern Europe', Public Management Review, 17(9), pp. 1305-1332.
Dascâlu, E. D., Marcu, N. and Hurjui, I. (2016) 'Performance management and monitoring of internal audit for the public sector in Romania', Amfiteatru Economic, 18, pp. 691-705.
Davey, K. and Péteri, G. (2006) 'Taxes, transfers and transition - adjusting local finances to new structures and institutions: The experience of the Czech Republic, Hungary and Slovakia' , Local Government Studies, 32(5), pp. 585-598.
Deming, W. E. (1993) The new economics. Cambridge: Massachusetts Institute of Technology, Center for Advanced Engineering Study.
Denhardt, J. V. and Denhardt, R. B. (2007). The new public service: Serving, not steering. Armonk-London: M. E. Sharpe.
Dragoç, D.C. and Neam^u, B. (2007) 'Reforming local public administration in Romania: trends and obstacles', International Review of Administrative Sciences, 73(4), pp. 629-648.
European Social Survey (2018) ESS Round 9 - 2018. Poland. Available at: https://www. europeansocialsurvey.org/data/country.html?c=poland (accessed 11 April 2021).
Fielding, N.G. (1988) Action and structure. Research methods and social theory. London: Sage Publications.
Gibbs, G.R. (2007) Analyzing qualitative data. Los Angeles, CA: Sage.
Kopec, K. (2020) Monitoring the quality of public services at the local level - Polish experience. In Fedorov, G., Druzhinin, A., Golubeva, E., Subetto, D. and Palmowski, T. (eds.) Baltic Region - The Region of Cooperation. Springer: Cham, pp. 259-265.
Kuczabski, A., Zastavetska, L. and Zastavetskyy, T. (2017) 'The reform of administrative division in Ukraine: Problems of territorial communities' formation in the Polish-Ukrainian borderland', Journal of Geography, Politics and Society, 7(2), pp. 87-97.
Lancaster, G. (2005) Research Methods in Management. Oxford: Elsevier.
Maksimovska, A., Stojkov, A. and Schmidt, P. (2015) 'Fiscal Decentralisation and ethnic politics in Macedonia', Lex Localis - Journal of Local Self-Government, 13(3), pp. 433-451.
Meyer-Sahling, J.-H. and Yesilkagit, K. (2011) 'Differential legacy effects: three propositions on the impact of administrative traditions on public administration reform in Europe East and West', Journal of European Public Policy, 18(2), pp. 311-322.
Michalski, T. (2014) Propozycja systemu monitoringu jakosci uslug publicznych na szc-zeblu lokalnym [A proposal for a system of monitoring of the quality of public services
at the local level]. In Ciok, S. and Janc, K. (eds.) Wspólczesne wyzwaniapolityki regional-nej i gospodarki przestrzennej, T. 2. Wroclaw: Instytut Geografii i Rozwoju Regionalnego UWr., pp. 93-103 (in Polish).
Nemec, J., Murray Svidroñová, M. and Éva Kovács, E. (2019) 'Welfare Co-Production: Hungarian and Slovak Reality', NISPAcee Journal of Public Administration and Policy, 12(2), pp. 195-215.
Neshkova, M. I. and Kostadinova, T. (2012) 'The effectiveness of administrative reform in new democracies', Public Administration Review, 72(3), pp. 324-333.
Norkus, Z. (2012) On Baltic Slovenia and Adriatic Lithuania: A qualitative comparative analysis of patterns in post-communist transformation. Vilnius: Apostrofa.
Public Service Commission South Africa (2008) Basic concepts in monitoring and evaluation. Available at: http://www.psc.gov.za/documents/guidelines.asp (accessed 21 May 2020).
Rose, R. (2009) Understanding post-communist transformation: A bottom-up approach. London - New York: Routledge.
Rosenbloom, D.H. (1993) Public administration: Understanding management, politics, and law in the public sector. New York: McGraw-Hill.
Shewhart, W.A. (1937) Statistical method from the viewpoint of quality control. Available at: http://library.isical.ac.in:8080/jspui/bitstream/123456789/6553/1/Statistical%20method%20 from%20the%20viewpoint%20of%20quality%20control.pdf (accessed 20 May 2020).
Silverman, D. (2014). Interpreting qualitative data. Los Angeles-London-New Delhi-Singapore-Washington: Sage Publications.
Struyk, R.J. (2007) 'Factors in successful program implementation in Russia during the transition: pilot programs as a guide', Public Administration and Development, 27(1), pp. 63-83.
Styrin, E. M. and Plaksin, S. M. (2012) 'Sistema gosudarstvennykh uslug kak obyekt monitoringa v Rossiyskoy Federatsii' ['The system of public services is an object of monitoring in the Russian Federation'], Public Administration Issues, 4, pp. 66-83 (in Russian).
Tikhomirov, Y.A. (2013) 'Gosudarstvennoe upravlenie: "vertikal" i "gorizontal" ['Public administration: "vertical" and "horizontal"], Public Administration Issues, 4, pp. 7-18 (in Russian).
Trutkowski, C. (ed.) (2011) Badania jakosci uslugpublicznych w samorzqdzie lokalnym [Research on the quality of public services in a local government]. Poznañ: Zwi^zek Miast Polskich (in Polish).
Vanags, E. and Vilka, I. (2006) 'Local government in the Baltic States: similar but different', Local Government Studies, 32(5), pp. 623-637.
Yuzhakov, V.N. (2014) 'Kachestvo gosudarstvennykh i munitsipal'nykh uslug: usiliya i rezul'taty administrativnoy reformy' ['The quality of public and municipal services: efforts and results of administrative reform'], Public Administration Issues, 1, pp. 52-72 (in Russian).
Zaufanie w sferze prywatnej i publicznej a spoleczenstwo obywatelskie [Trust in the public and private sphere and the civic society] (2006). Warszawa: CBOS (in Polish).
The article was submitted: 22.05.2020; approved after reviewing: 14.12.2020; accepted for publication: 08.11.2021.
