CC BY
1Budget Program Evaluation in Armenia as a Tool to Improve the Effectiveness of Public Spending
Chapanyan Taguhi S.
PhD student of Public Administration Academy of the RA (Yerevan, RA)
https://orcid.org/0009-0007-9759-6558 taguhichapanyan@gmail.com
UDC: 336.1; EDN: ZWXANI; JEL: H50, H61; DOI: 10.58587/18292437-2024.1-188
Keywords: budget expenditure, program budgeting, evaluation, monitoring
PjnLghmwj|h ö|iiuqjih|i|i qiiiuhiiiiniiLil|i luu|iiiiiiniiiiiiiLil ii|iii|hu luiiii|iiii||iii öiii|iiuh|i|i
iu|ii]jiiLÎiiiii|hiniiLpjiuii piii|ih|uii[iliuii qnpö|g
QшфшhJшh ßwqnthfr U.
^щшитшй^ щЪтш^шй ^шпш^шрйшй ш^шцЫ^щ^ шищ^ршйш (Ьркшй,
ийфифинфр. 4hp2^ü шшийш^ш^шй, ^й^щЬи ш2faшрh^ 2шш hp^pühpniü, щкщЬи tl -ицшитшйтй, ^рш^шйшдф1 hü pjnLghrnuijfrü qqшф ршрЬфп^тйЬЬр: Ujq ршрЬфп^тйЬЬр^ ^шркпр йшир pjnLghrnuijfrü 0ршдрЬр^ qüшhшщüшü hшüш^шрф йЬрцртйй t: и^шшшйр^й рййшрЩ^й hü -ицшитшй^ pjnLghrnuijfrü hшüш^шрф ^Ьц^рйЬрр к qüшhшшüшü hшüш^шрф йЬрцрйшй ^шркnрnLpJnLÜр, ^щЬи йшк ц^тшрЩ^й hü щи hшüш^шрq^ йЬрцрйшй шдрйшршйрщфй ünцhlühрр: Uju^un^ hnq^öft Ьщшшш^Ь t й2ш№1 шЬиш^шЬ, ühpnqш^шü, ^щЬи йшк qnрöüш^шü шnш2шрUnLpJnLÜühр ---nLÜ pjnLghrnuijfrü öршqрhрfr qüшhшшüшü hшüш^шрq^ йЬрцрйшй фршрЬр^! ^щЬи щЬшш^шй шрцJnLÜш4hщnLpJшü
ршрйршдйшй q^öftg: -ш2пд4ш0 ф^йр титййши^рЬрф hnц4шöр ф^йш-й t qnрöüш^шü щшт^Ьрш-дшййЬр шпш2шр^1 gшцшgш^шünLpJnLÜ Ü2ш4nцühрk щЬшш^шй щш2шnÜJшühр^ к 2шhшqрqfrn ^ПД^ЬР^ hшüшр, uiju^un^ hnq^önLÜ рЬвд&Цтй t pjnLghrnuijfrü öршqрhрfr qüшhшшüшü ^uijnLÜ к ршфшйд^^ hшüш^шрq^ ^шр^ртр^йр hш24hш4nдш^шünLpJшü, пЬишрийЬр^ ощш^йш^шдйшй к -шщишшй^ püш^¿nLpJшüр 2П2шФЬ]^ oqnLШÜhр^ шщшhn4üшü q^önLÜ: -пцЦш0р йшк шйцршцшпЬшй t qüшhшшüшü hшüш^шрф йЬрцрйшй йшршшЬрш^ЬрйЬр^й, frü^^ufrg hü щфшфЬр^ ишШшйшфш^тййЬрр, ^йиш^щ^д^йш! ^шрnдnLpJnLÜühрр к 2шhшqрq^n ^ц^Ьр^ ühрqрш44шönLpJnLÜр: -<ш^^дшршпЬр' pjnLghrnuijfrü 0ш^иЬр, &ршдршфй pJnL2hшш4nрnLÜ, qüшhшшnLÜ, ün^p^^üq
Оценка бюджетной программы армении как инструмент повышения эффективности
государственных расходов
Чапанян Тагуи С.
Аспирант Академии государственного управления Республики Армения (Ереван, РА)
Аннотация. В последнее десятилетие, как и во многих странах мира, а также в Армении, произошли значительные бюджетные реформы. Важной частью этих реформ является введение системы оценки бюджетной программы. В статье обсуждаются проблемы бюджетной системы Армении и важность внедрения оценочной системы, а также рассматриваются альтернативные модели внедрения этой системы. Таким образом, целью статьи является разработка теоретических, методических, а также практических рекомендаций по внедрению системы оценки бюджетных программ в РА как инструмента повышения эффективности государственных расходов. Изучая успешный опыт, статья стремится предложить действенные идеи для политиков, государственных чиновников и заинтересованных сторон, таким образом, в статье подчеркивается важность стабильной и прозрачной системы оценки бюджетных программ в обеспечении подотчетности, оптимизации ресурсов и предоставлении ощутимых выгод населению Армении. В статье также рассматриваются проблемы внедрения системы оценки, такие как ограничения данных, институциональный потенциал и взаимодействие с заинтересованными сторонами.
Ключевые слова: бюджетные расходы, программное бюджетирование, оценка, мониторинг
Introduction. The existence of an efficient budget system contributes to the more targeted distribution of budget funds. The evaluation of budget programs stands as a fundamental mechanism in assessing the efficiency, effectiveness, and overall impact of government spending initiatives. An effective evaluation
framework for budget programs not only ensures accountability and transparency but also empowers policymakers with the necessary insights to steer resources toward areas that yield optimal outcomes.
This article aims to analyze the need and potential opportunities for implementing a budget program evaluation system for improved
governance, better policymaking, and improved service delivery. Moreover, understanding how the budget program evaluation system works in other countries can be valuable insights and lessons for Armenia, as Armenia is still in the early stages of implementing this system, and studying the challenges faced by others can be valuable in avoiding such problems. Thus, by examining the methodologies used, the successes achieved and the obstacles encountered, this article seeks to highlight the key role that evaluation mechanisms play in the more effective and efficient allocation of public resources.
Literature review. An important prerequisite for the implementation of the budget program evaluation system is the introduction of new budget classifications, such as performance budgeting. Performance budgeting has different forms and program budgeting is one of them. Program budgeting is not only the most widespread form of performance budgeting, but it is the form of performance budgeting most applicable to government budgets as a whole. Program budgeting aims to structure the budget in such a way that facilitates good expenditure prioritization, while sim ultaneously placing increased pressure on ministries and agencies to improve the effectiveness and efficiency of their expenditures [21, p.9].
The logic of program budgeting directly connected with evaluation system. Thus according to Robinson (2007), evaluation is an essential tool for good budgeting and a core element of any well-designed government-wide performance budgeting system.
Evaluation can support the budget process by helping either to:
1. Identify programs or components of programs which can potentially be cut: this means programs that are not cost-effective and which cannot readily be made cost-effective through policy design or management changes,
2. Identify savings that can be made by improving the efficiency of service delivery [22, p. 42].
The objective of evaluation is:
a) To determine as systematically and objectively as possible the relevance, efficiency, effectiveness and impact of the Organization's activities in relation to their objectives.
b) To enable the Secretariat and Member States to engage in systematic reflection, with a view to increasing the effectiveness of the main programmes of the organization by altering their content and, if necessary, reviewing their objectives. It analyses the level of achievement of both expected and unexpected results by examining the
results chain, processes, contextual factors and causality using appropriate criteria [5, p 6].
Before analyzing evaluation systems, we must first define the essence of evaluation, as well as introduce the definition of monitoring, which is connected with evaluation. According to the OECD DAC evaluation is the systematic and objective assessment of a planned, ongoing or completed intervention, its design, implementation and results [6, p. 12]. With other definition, evaluation is an objective, independent and systematic examination of the extent to which a program or project has achieved (or is achieving) over time its stated objective and, therefore, is meeting the needs and priorities of Member States [13, p. 6]. According to the "concept of implementation of the evaluation system of RA state budget programs", evaluation is defined as a systematic, impartial study-evaluation of the importance, implementation process and results of the beginning, ongoing or completed budget program, event or policy structure. And what about monitoring, it is defined as a continuous process that involves the systematic collection or collation of data (on specified indicators or other types of information). Provides the management and other stakeholders of an intervention with indications of the extent of implementation progress, achievement of intended results, and occurrence of unintended results, use of allocated funds and other important intervention and context-related information [6, p. 12]. In other definition, monitoring is a continuous function to inform the program or project managers and stakeholders of progress achieved against planned results (outputs, outcome and objectives) [13, p. 6]. The UN gives very simple definition of these terms, according to which: "Monitoring means keeping track of what you are doing while you are doing it, so that you can take corrective action if necessary.
Evaluation means finding out if you have achieved the effect on your target population that you said you would achieve, after you have finished implementing the activities." [14, p. 8].
In practice sometimes the difference and borders between evaluation and monitoring is not so clear. However, they work together in synergy. Monitoring information is a necessary but not sufficient input to the conduct of rigorous evaluations. While monitoring information can be collected and used for ongoing management purposes, reliance on such information on its own can introduce distortions because it typically covers only certain dimensions of a project's or program's activities, and careful use of this information is needed to avoid unintended behavioral incentives. In contrast, evaluation has the potential to provide a more balanced interpretation of performance. But
evaluation is a more detailed and time-consuming on monitoring information to identify potential activity, and because of its greater cost it needs to be problem issues requiring more detailed investigation conducted more sparingly. One approach is to rely via an evaluation [7, p. 20].
Table 1. The difference between monitoring and evaluation
Monitoring Evaluation
Frequency Periodically, regularly. One time, at certain stages.
Focus Follow the progress, control the processes, work plans. Examine the implementation of intermediate and final results from direct results according to evaluation criteria and evaluation questions.
Basic purpose Improve event results in line with the work plan, make ongoing improvements Improve program effectiveness and impact, contribute to future planning, learn lessons for similar events
Methodology Analysis of current reports, current field observations, rapid appraisal with quantitative or qualitative data collection. Targeted data collection using surveys, mixed methods research, experimental and quasi-experimental methodologies.
Executors Project managers, community workers, beneficiaries, donors, etc. External evaluators and evaluation officers with the participation of project managers, donors, beneficiaries, etc.
Thus, monitoring and evaluation are two different types of exercise in the project
management cycle, although they are designed to complement each other, but they should be carried out separately in order to have unbiased assessment
on the implementation of programs and projects. Therefore, in the PMP they both must be dealt separately because frequency, approach, methodology and scope are different for both exercises [15, p. 8].
Investigation
Table 2. Other control terms:
•Detect irregularities, waste, illegal or criminal activities
Audit
Research
•Risk management, verification of compliance of financial and other funds expenditure
•Financial audits check the reliability of the organization's accounts
• Fntemal audit assesses an organization's internal controls, including its corporate governance and accounting processes
•A systematic process of collecting and analyzing data and information to generate new knowledge, answer a specific research question, or test a hypothesis
Study, including assessment
• Summary and interpretation of current knowledge information and data on a specific topic: simple assessment, rapid appraisal, situational analysis, document analysis, mapping, survey and questionnaire study, including simple assessment
•A procedure by which the credibility of the reports, declarations, calculations of taxes and other mandatory payments, baseline data, and other documents submitted by the entity is determined and their compliance with the requirements of laws and other legal acts
Inspection
Thus, in summary, we can state that monitoring is a regular review of performance indicators to find performance issues and trends, and evaluation is an in-depth analysis using formal evaluation methodologies.
In addition to the close connection between evaluation and monitoring, other different control terms are used in practice, which should also be clearly defined and separated from evaluation.
Thus, during the implementation of the evaluation system, each country must define the term evaluation based on the goals that the system pursues, and this definition must be clearly specified.
Research. Armenian budget system coordinated by the Law of the Republic of Armenia "On the Budgetary System" was adopted in 1997, and within the years this low has changed periodically, thus, one of the main change was the new classification of budget expenditure, thus, since 2019, the RA state budget is drawn up in program format. It enables decision-makers to move towards a goal by comparing costs and results, to prioritize costs and to increase the efficiency of public expenditure management. The best way to improve the level of public expenditure management is to clearly define objectives, which can be considered the biggest challenge of program budgeting. As a result of the implementation of program budgeting,
the efficiency of the use of public money is increased, the "sectoral strategy budget" connection is improved and the government's accountability in terms of achieving (or not achieving) results [18]. According to the program budgeting approach, the RA budget is made up of programs, which are made up of measures. The object of budgeting and financing are measures, all measures necessarily contain non-financial results (performance) indicators, the latter is very important for program classification. They are mainly of two levels.
• indicators of direct results - are the indicators characterizing the implemented measures. Typically service volume and/or service or product quality indicators,
• Indicators of final results - are indicators that can be used to track changes in the public environment.
In 2022 state budget expenditures actually amounted to 98% compared to the planned. 174 programs were planned in the RA 2022 state budget, which, as a result of subsequent adjustments, made 175, of which actual funds were allocated for 171, moreover, 944 of the 977 measures planned within the programs were fully or partially implemented, i.e. the performance of the planned measure is 96.6%.
700 600 500 400 300 200 100 0
Figure 1. Expenditures of the RA state budget according to responsible bodies (billion AMD) [19]
✓ / / ✓ J? ./ J
/
101
100
99
98
97
96
95
94
93
92
91
2022 planned 2022 actual Performance
As can be seen from Figure 1, the planned expenditures of the RA state budget according to the responsible bodies actually had a certain deviation, in particular, the increase of the planned index was mainly made in the expenses of the RA Ministry of Defense, as well as the RA Ministries of Labor and Social Affairs and the RA Ministry of Health, and the main part of the reductions was made in the
expenses of RA Territorial Administration and Infrastructures, RA Education, Science, Culture and Sports and RA Finance Ministries. The same trend is also observed in the case of the execution of expenses for separate programs approved by the 2022 state budget of the Republic of Armenia, for instance "Preservation and exhibition of natural samples" program expenditure performance is 100%
against the specified plan; and the performance of the "Promotion of Plantation and Plant Protection" program was 18.7%, that is, less financial resources were allocated than planned. While there are still many areas that are unlikely to develop quickly without the effective intervention of the state, for example, it is unlikely for producers in Armenia to develop a comparative advantage in goods with higher implicit income levels [10, p. 79]. Thus, budget program evaluation is crucial for achieving budget goals, it helps identify which programs are most efficient and effective in achieving their objectives, by assessing the impact of various budgetary allocations, policymakers can allocate resources to programs that deliver the best results. The RA Ministry of Finance publishes quarterly and annual reports on state budget implementation, where state budget programs and measures are presented with their planned and actual performance indicators, but there is no empirical evidence-based evaluation to evaluate the results of the programs. However, by understanding the causes of poor outcomes, policymakers can make informed decisions about how to adjust and optimize program design, implementation, and resource allocation to
Thus, World bank research also highlighted there are significant deviations between allocations within medium-term expenditure frameworks, annual budget plans, and actual execution, particularly for capital expenditure. Comparison between MTEF(s), annual budgets, and execution highlight how Armenia has faced persistent challenges with under-execution of capital spending [1, p. 28]. According to Mkrtchyan, despite all the measures implemented in the field of implementation of program budgeting, the Republic of Armenia still needs significant institutional mechanisms for the implementation of a long-term strategy for the implementation of program budgeting [12, p. 115]. And despite all the goals pursued by the introduction of program budgeting, the gaps that still exist in this area: weak connection between strategic documents and budget programs, lack of clear criteria for setting priorities, non-
improve future performance. Evidence-based decision-making ensures that budget allocations are based on concrete data rather than assumptions or political considerations.
83% of the state budget expenses were allocated to current expenses, 17% to operations with non-financial assets. Although the 2020-2021 capital expenditure shortfall eased in 2022, performance against the revised plan was 94.9%, an improvement of 2.4 percentage points from the previous year's 92.5%, however, the problem of increasing the efficiency of management and implementation of capital programs remains relevant, moreover, it is noteworthy that the performance of programs implemented with external support (without co-financing from the state budget) has worsened, making 80.5% of the specified program. Clearly, planned capital spending reallocations often occur in the wake of economic downturns, though, "Armenian authorities have defined by law, that capital expenditures cannot be used as a tool for fiscal downward adjustments, as new debt cannot be used for public consumption" [9, p. 48].
19]
targeted performance indicators of programs and measures, lead to a serious demand for the monitoring and evaluation of the state budget implementation, and the implementation of the budget program evaluation system is considered an important step in this direction. Back in 2010, during the study of RA Public Financial Management Reform Priorities by the World Bank, it was emphasized that the non-financial information in the budget bids is not yet being put to full use during the budget deliberations. The type of information that could be included would relate to the outputs and outcome indented by the funding programs. As the solution of raised issues it is important to complement the incorporation of performance information during the upstream budget preparation with downstream performance reporting, and robust monitoring and evaluation
Table 3. Capital expenditures of the RA state budget according to funding sources (billion AMD)
2022 planned 2022 actual Performance (%)
Expenditure on non-financial assets (capital expenditure), including: 410.1 389.1 94.9
Program implemented with external support (including co-financing by the RA government) 66.2 54.0 81.6
Programs implemented at the expense of internal resources 343.8 335.1 97.5
systems that periodically assess program performance [20, p. 19].
The evaluation of budget programs implies a systematic way of obtaining, analyzing and using information about the program, the purpose of which is to answer questions about the effectiveness and usefulness of the program. For this purpose, in 2022, the Prime Minister's decision N 446-A approved the "Concept of implementation of the State Budget Program Evaluation System". The establishment of a program evaluation mechanism in Armenia is an important element for supporting the ongoing public finance management reforms and improving the budgeting process as well as enhancing the effectiveness of public spending. To develop and strengthen performance monitoring and evaluation practices and institutions, the OECD focuses on a systemic approach, which looks at 3 main pillars:
• Promoting the impact of performance data and policy evaluations by embedding their use into decision-making processes, fostering demand for evidence, as well as communicating and disseminating results;
• Promoting the quality of performance data and policy evaluations by investing in public sector skills and capacities, developing quality assessment and control mechanisms, and adopting standards.
Table 4.
• Building an institutional framework by putting the right legal, policy and organisational measures in place to support the performance of public policies [16].
The formulation of a program evaluation should take into account the following key question:
• Should the system be centralized or decentralized?
• Should the evaluation be performed in-house or outsource?
• Should the evaluation unit perform ex-ante or ex-post evaluation?
• Why are we conducting the evaluation? What specific decisions do we aim to make based on its outcomes?
• What types of information are crucial for making these decisions or enlightening our target audience?
• When must we collect this information? What is the deadline for obtaining the necessary data?
• Who is the intended audience for the evaluation? Who will benefit from the insights and findings?
Both centralized and decentralized evaluation systems have their advantages and disadvantages, and the choice often depends on the specific goals, context, and characteristics of the organization or government implementing the evaluation process.
Centralized (Establishment of a structural unit under the Ministry of Finance, which will be responsible for assessing the programs / sub-programs of the Ministries / Budgetary Organizations.) Decentralized (Establishment of a structural unit in each Ministry and agency, which will be responsible for assessing programs/subprograms of budget.)
Advantage 1. Centralized evaluation ensures a standardized and consistent approach to evaluation criteria, methodologies, and reporting across different programs or departments. 2. Decisions made in a centralized environment, 3. Streamlined implementation of policies and initiatives, and control over the strategic direction of the organization, 4. Can be implemented in a Short period of time, 1. Decentralized evaluation allows for a more tailored approach, considering the unique characteristics of individual programs or ministries. 2. Availability of information, 3. Quick decision and response times, 4. Skilled and/or specialized employees, 5. Evaluation of a large number of programs/sub-programs,
Disadvantage 5. Limited opportunities to provide feedback and a higher risk of inflexibility, 6. Difficulties in receiving information, 7. Impossible to evaluate all the programs/sub-programs, 8. Interference with the work of another body, 1. 1. Expansion of the number of staff in state bodies and large expenses, 2. Long duration of system formation, 3. Lack of interest in raising ineffective programs,
An example of a centralized evaluation system is Chile, since 1997 the government has strongly emphasized evaluation, where the DIPRES (Budget Directorate, Ministry of Finance) is the main actor. By contrast with the centralized Chilean approach, t he new Canadian evaluation policy leaves evaluatio n as an essentially decentralized process. Both the m anagement of specific evaluations and the choice of evaluation topics in any specific year remain essenti ally matters for the spending ministries themselves t
0 decide. Most evaluations are carried out by interna
1 evaluators within spending ministries, although out side evaluation experts support some evaluations [11, p. 36].
An ideal evaluation system would combine both centralized and decentralized evaluation. In such a combined system, the Evaluation Unit should have the additional function of promoting and regulating decentralized evaluation. For example, the Ministry of Finance of Netherlands the is the main actor caring out evaluation, but all ministries are responsible for carrying out policy evaluations in their area.
It is envisaged that the evaluation system of RA state budget programs should be centralized under the direct supervision of the RA Prime Minister. It is required as a long-term perspective, after the introduction of centralized evaluation, during the time Armenia pass to combine evaluation.
A choice has to be made as to whether evaluations will be mainly done in-house or outsourced.
♦ In-house option: it is in principle possible to build up a group of people in the Evaluation Unit capable of carrying out evaluations based on the practical methodologies. In-house option supposed expansion of staff and required capacity building.
♦ Outsourced option: this would involve contacting out evaluations. The private company is an independent, it will not have any subjective interest in the achieved results of the program/subprogram and on the target groups in their evaluation process. It provides an opportunity to attract more highly qualified professionals.
However, as there are not private specialized companies in Armenia, thus in case of Armenia suggested evaluations done in-house.
The decision of whether an evaluation unit should perform ex-ante or ex-post evaluations depends on the goals and objectives of the evaluation, as well as the specific context of the program or policy being assessed. Both types of evaluations serve different purposes and provide distinct insights: Ex-ante evaluations refers to the systematic appraisal of new spending proposals. This type of evaluation focuses on predicting and analyzing the expected outcomes, costs, and benefits
before resources are committed to the initiative. Exante evaluations are valuable for decision-makers as they provide information that can guide investment decisions, policy design, and resource allocation. Ex-post evaluation refers to evaluations of spending that is ongoing or completed. This type of evaluation focuses on what was achieved in practice, examining the results. Ex-post evaluations are crucial for learning from experience, improving future interventions, and holding accountable those responsible for the implementation.
In many cases, a comprehensive evaluation strategy may include both ex-ante and ex-post elements. Here are a few considerations when deciding which type of evaluation to prioritize or whether to use a combination:
• budget constraints and timelines,
• the primary goal, if it is to inform decision-making for future programs, ex-ante evaluations may be more relevant, and if it is to assess the performance and impact of a completed initiative, ex-post evaluation is necessary.
The evaluation should inform Government whether the program:
• is achieving the intended results;
• should continue and/or be improved;
• may be better delivered by an alternative
service provider;
• should cease.
Evaluation activity should be aligned with the annual Budget process, this means to provide effective feedback on the decision-making of public policy makers. This involves using evaluation results during the budget process, such as evaluating new initiatives through rapid appraisal at the budget application stage, and in the case of ongoing programs, supporting proposals to continue, extend or expand existing funding. Evaluation benefits for the government, agencies, public servants and the community. But, meanwhile performance information cannot be used mechanically in reaching budget decisions: The use of performance information in decision-making processes also needs to consider other information categories, such as policy-program priorities and restrictions imposed by fiscal policy [11, p. 46].
According to OECD there are 6 criteria's; relevance, coherence effectiveness, efficiency, impact and sustainability [2, p.18]. Each of the six criteria is summarized by a broad question, which illustrates its overall meaning. Each one represents an important element for consideration:
• Relevance: Is the intervention 1 doing the right things?
• Coherence: How well does the intervention
❖ Effectiveness: Is the intervention achieving its objectives?
❖ Efficiency: How well are resources being used?
❖ Impact: What difference does the intervention make?
❖ Sustainability: Will the benefits last?
The evaluation criteria's purpose is to support
consistent, high-quality evaluation within a common framework. For each evaluation can be used different criteria's, or combination of them.
There are a wide range of evaluation methodology from simple rapid appraisal to impact
Table 5. Benefits of evaluation [3, p. 5]
Stakeholder Potential benefits
Government • Information to assist decision making. • Improved ability to achieve government priorities. • Efficient resource allocation. • Highlights achievements and opportunities to strengthen performance. • Encourages greater public trust in government.
Agencies • Stronger basis for informing government priorities and resource allocation. • Improved service delivery and client satisfaction. • Builds an agency's reputation for innovation and continuous improvement.
Public servants • Develops new skills and broadens experience. • More opportunity to shape public policy. • Fosters a more dynamic and creative work environment. • Recognizes and rewards efforts to improve performance.
Community • Better government services. • Informative government reporting. • Transparent and accountable government. • Public monies used more efficiently. • Greater confidence in activities of government.
evaluation. In general, the selection of methodology should consider the resources available for evaluations, both organizational and financial in nature, as well as professional abilities; the amount of time each program will need for its implementation; previous evaluations available; and when the information will be needed. In Chile, initial evaluations were rapid or desk evaluations, using a simple methodology that allowed for developing abilities, making it possible to apply more complex methodologies later [11, p. 44].
As mentioned earlier, according to the "concept of implementation of the evaluation system of RA state budget programs" evaluation system in Armenia should be centralized, evaluations will carried-out in-house. Lessons from Chilian and Australian experience, support the idea to create centrally-driven evaluation system, thus, this is a right decision at this moment, because frhp^mjnLtfu there is no capacity for outsource, and at the same time, there is also a severe shortage of personnel with appropriate qualifications, so at this stage it is advisable to create a centralized structural unit that will contribute to the development of the sector in the public sector, the acquisition of experience, the formation of qualified personnel, which will create the basis for a transition to a decentralized or of the combined model. At the same time, intensive training of personnel is very important for the formation of an effective system.
According to the concept, it is envisaged that ex-ante and ex-post evaluations should be carried out by the same entity. However, it should be noted that it is important to understand that the evaluation unit cannot evaluate everything, first of all it doing both type of evaluation means overloaded the unit. Also, the concept envisages that after a certain stage, all new initiatives should undergo preliminary assessment. But it would be impossible to evaluate every new spending proposal during the budget preparation process, and the one unite can carried-out only part of ex-ante evaluation, or ex-ante should be assigned to another unit. Thus, in the formation of an effective system, it is important to clarify roles and responsibilities of all stakeholders.
The concept envisages that all state budget programs will be evaluated over the next ten years, however it would be impossible for a to evaluate everything. It is true that certain countries have had policies of evaluating all programs over a cycle of
three or five years, for example in the past in Canada and Australia. However, such an approach can only work if most evaluations are done on a decentralized basis, for example the case of Chile demonstrates that it is possible to initiate a system for evaluating individual programs, with between five and eight evaluations each year [11, p. 44]. And other cases, there are programs, which don't have meaning to evaluate, such as administrative programs or a school construction program that does not have a result indicator.
A strategic approach to evaluation should be taken and understanding that the formation of an effective system requires a long time, in case of Australia, it took several years to achieve cultural change: required strong leadership, focused recruitment/promotion, on-the-job training, staff turnover [17, p. 18].
Studying the "concept of implementation of the evaluation system of RA state budget programs", we identify a number of methodological approaches that will hinder the formation of an effective system, therefore, taking into account the lessons learned from the experience of other countries, evaluation methodology is a key determinant of the quality of evaluation [8].
Conclusion and recommendations:
Based on the provided material and drawing insights from the other countries experiences, here are some recommendations:
1. The evaluation of budget programs is a potentially important reform in the ongoing struggle to increase the efficiency of public spending. Rather, it is a reform that can be expected to take at least a few years to implement in its initial form.
2. To establish a centralized evaluation system in Armenia, the concentration, at least in the initial stages, will contribute to the development of the public sector, the acquisition of experience, and the formation of qualified personnel.
3. Emphasize the importance of intensive training for increase the evaluation skills for staff. Achieve this by developing and implementing a training strategy to provide basic evaluation skills and more practical guidance.
4. Clearly define roles and responsibilities of all stakeholders involved in the evaluation process to avoid overloading the evaluation unit and ensure efficiency (in case of ex-ante evaluation).
5. Recognize the importance of the evaluation methodology as a key determinant of the quality of evaluation. In Armenia evaluations should be developed gradually in terms of selecting the evaluation targets, the methodology, corresponding operational processes.
6. Emphasize, not all state budget programs may need the evaluation. Consider a flexible
approach, where the scope, methodology and depth of evaluation may vary based on the nature of the program and its impact.
References:
1. Armenia public expenditure review: improving spending efficiency, WB, October 2023, https://goo.su/GnkME (Ссылка сокращена).
2. Applying Evaluation Criteria Thoughtfully, OECD, Paris, 2021. https://goo .su/CUdPcbo (Ссылка сокращена).
3. Australian Capital Territory Government 2010, Evaluation Policy and Guidelines, (December 2010), Policy Division, ACT Chief Minister's Department.
4. Decision N 446-A of the RA Prime Minister of April 28, 2022 "On approving the concept of implementation of the evaluation system of state budget programs" [in Armenian]
5. ESCAP Monitoring and Evaluation Policy and Guidelines, 2023 edition, https://goo.su/dmnI (Ссылка сокращена).
6. Glossary of Key Terms in Evaluation and Results-Based Management 2nd Edition - Prepublication version, 2022 https://one.oecd.org/document/DCD/DAC/EV(2022 )2/en/pdf
7. Glossary of Key Terms in Evaluation and Results Based Management, OECD, Paris, 2002, http://www.oecd.org/dataoecd/29/21/2754804.pdf
8. Government of Canada https://goo.su/zc5nk (Ссылка сокращена).
9. Grigoryan K., Petrosyan G., Vardanyan K., Avagyan G., Assessment of the effects of public investment on GDP growth: case of Armenia. Sciences of Europe. No. 78, Vol. 2, 2021. p. 4. DOI: 10.24412/3162-2364-2021-78-2-46-60. Available at: https://u.to/8SVTIA (Ссылка сокращена).
10. Grigoryan, Karen, Study of the Peculiarities of Export Developments in EU Member Countries and in Armenia, (September 6, 2012). Romanian Journal of European Affairs. Vol. 12, No. 3. September 2012. pp. 65-82. Available at SSRN: https://ssrn.com/abstract=2142370 or http://d x.doi.org/10.2139/ssrn.2142370
11. Guzmán Marcela, Irarrázaval Ignacio, and Boris de los Ríos, Monitoring and Evaluation System: The Case of Chile 1990-2014, 2014 https://ieg.worldbankgroup.org/sites/default/files/Da ta/reports/ecd wp 29 Chile.pdf
12. Mkrtchyan Edgar, Harosyan Sveta, Hakobyan Gor, "Key Aspects of Program Budgeting Implementation in the World and in the Republic of Armenia," // Finansovyj zhurnal — Financial Journal, Financial Research Institute, Moscow 125375, Russia, issue 4, pages 115-120, 2018
13. Monitoring and Evaluation Guidelines, Technical Cooperation Projects https://www.iaea.org/sites/default/files/documents/tc /TCP MandE Manual.pdf
14. Monitoring and Evaluating Youth Substance Abuse Prevention Programmes, UNODC and WHO, Vienna, 2006
15. Monitoring & Evaluation Framework, UNISDR, https://www.preventionweb.net/files/49324 unisdrm eframeworkverl.O.pdf
16. Monitoring and evaluation, OECD, https://www.oecd. org/governance/budgeting/monito ring-and-evaluation/
17. Monitoring and Evaluation (M&E) Systems to Improve Government Performance, presentation to the PEMPAL Plenary Conference Istanbul, Keith Mackay Independent Evaluation Group The World Bank, 2008 https://www.pempal.org/sites/pempal/files/attachme nts/1-mackayeng.pdf
18. Poghosyan, S., Minasyan, G., Program budgeting reforms in Armenia: Analytic toolkit. GIZ. Yerevan, 2014 [in Armenian]
19. Report on the State Budget of the Republic of Armenia" for 2022 [in Armenian] https://www.minfin.am/hy/page/petakan byujei has hvetvutyun 2022 t tarekan
20. Republic of Armenia Public Financial Management Reform Priorities, WB, 2010, https://u.to/LyZTIA (Ссылка сокращена).
21. Robinson, M., "Performance Budgeting: Linking Funding and Results", IMF, Washington, 2007
22. Robinson, M., "Performance-based Budgeting manual", CLEAR Training Materials
Сдана/^шй&йф11 01.02.2024 Рецензирована/^рш^пиф11 13.02.2024 Принята/£йщьйф11" 20.02.2024
