CC BY
12DOI: 10.5281/zenodo.1051164 UDC: 614.2:616.1/.8-084(478)
Management of Delphi study regarding the development of ongoing behavioral risk factor surveillance system in the Republic of Moldova
*Raevschi Elena1, Ababii Ion1, Vataman Eleonora2
'Nicolae Testemitsanu Department of Social Medicine and Health Management Nicolae Testemitsanu State University of Medicine and Pharmacy, Chisinau, the Republic of Moldova 2Institute of Cardiology, Chisinau, the Republic of Moldova Corresponding author: elena.raevschi@usmf.md. Received August 14, 2017; accepted October 01, 2017
Abstract
Background: Implementation of ongoing behavioral risk factor surveillance system (BRFSS) can improve prevention and control of noncommunicable diseases (NCDs) in the Republic of Moldova. Delphi study regarding the development of ongoing behavioral risk factor surveillance system in the Republic of Moldova contributes to evidence-based credible model of implementation in order to support National Strategy on Noncommunicable Diseases Prevention and Control for the period 2012-2020. The article has highlighted the aspects of designing Delphi survey which should be considered for making sense of consensus.
Material and methods: The study used Delphi techniques to achieve in an effective way experts' consensus in terms of development of ongoing BRFSS in the Republic of Moldova.
Results: The research found that Delphi panelists similarly trained and competent in the area of knowledge related to the target issue contributed to achievement of the consensus agreement for all items in discussion regardless the national or international affiliation.
Conclusions: Well-designed Delphi panel are important for effective and reasonable survey management in terms of achieving feasible consensus agreement to support effective decision-making for further development. Key words: Delphi study, ongoing behavioral risk factor surveillance system.
Introduction
Ongoing surveillance of behavioral risk factors is a powerful tool in reducing mortality from noncommunicable diseases [1, 2]. Proportional mortality from noncommunicable diseases is 89% in the Republic of Moldova [3]. There is no system for ongoing surveillance determining the interventions priorities regarding behavioral risk factors in the Republic of Moldova.
The objective of the study was to determine experts' consensus on a variety of national and international health policies in terms of development ongoing Behavioral Risk Factors Surveillance System in the Republic of Moldova.
Delphi method is a procedure that allows a group of experts to participate jointly but anonymously by many rounds of questionnaires, in order to reach consensus for forecasting, planning or for strategies development questions [4]. The Delphi survey has been performed in order to provide good practice evidences for potential implementation of behavioral surveillance system in the Republic of Moldova, adjusted to the local needs supporting the National Strategy on Noncommunicable Diseases Prevention and Control for the period 2012-2020 [5].
Material and methods
The Delphi survey was performed for a period of 8 months, from July 2016 till February 2017. The Delphi study included two written Delphi rounds to complete a questionnaire via e-mail. The selection was based on including national and international professionals in the field of management of risk factors surveillance, using snowball-sampling design. Invited international experts were from countries with ongoing surveillance system based on BRFSS U.S standards. Delphi panel included 19 public health experts from USA, Italy, Romania and the Republic of Moldova, who ac-
cepted the invitational letter sent individually to the list of 55 potential participants.
The first questionnaire was prepared based on the issue areas of behavioral risk factors control and prevention at the national and international level [5, 6, 7]. The questions content was a result of many different sources, as following: Action Plan for the period 2016-2020 regarding the implementation of National Strategy on noncommunicable diseases prevention and control 2012-2020 [8], Action Plan for the period 2014-2020 regarding the implementation of National Program on cardiovascular diseases prevention and control for the period 2014-2020 [9], research findings from pilot cross-sectional telephone survey test conducted in the Municipality of Chisinau of the Republic of Moldova (n=800) mainly based on the US BRFSS's protocol [10], and review of recent publications on national ongoing behavioral risk factor surveillance systems [11, 12, 13]. A 9-point Likert agreement scale was used to measure the strength of the experts' consensus with a clear statement (from 1 being completely disagreed to 9 being completely agreed). The core part of the first round questionnaire (38 items) was structured around the US BRFSS's protocols [1] and tasks regarding the evaluation of a public health surveillance system mainly based on Updated Guidelines for Evaluating Public Health Surveillance Systems [14], as follows: purpose (9 items), operation (15 items), and attributes (14 items) of the surveillance system.
Pilot round questionnaire was performed with participation of three experts not belonging to the written Delphi panel.
The questionnaire for the second round was based on the results and comments of the first round.
The statistical analyses were performed using Excel for Mac 2011(version 14.7.2) and PSPP (version 0.8.4) which is free software application for analyses data, intended as free alternative for IBM SPSS Statistics. The five number sum-
54 -i
maries were used to present the information regarding the consensus of the Delphi panel experts. Boxplots graph presentation was used to display five number summaries: the ends of whiskers are the minimum and maximum values, the bottom and top of the box indicate Q1 and Q3 respectively, and the median is displayed by the band inside box. A Fisher's Exact test was applied to assess the significance between two independent groups. The significance level was set at a 2-sided of .05
A criterion for achieved consensus of the experts on a nine point Likert rating score is considered if the median has to be at seven or higher.
Results
All experts who accepted the invitation to join the Delphi study panel completed the first round questionnaire consequently no dropout rate was registered. The first round Delphi survey included 19 participants with 9 international and 10 national experts.
The strength of the experts' consensus for all items included in the first questionnaire compartment of the purpose (9 questions) of Behavioral Risk Factor Surveillance System development in the Republic of Moldova was at the median score from 8.5 to nine points. The items related to the operation (15 questions) of the surveillance system were at the median score from seven to nine (fig. 1).
[-
£ 8 1 - -
a
■z
E J
aj .-,
|
i 2 3 4 5 6 V 8 9 10 11 12 13 14 IS 16 17 18 19 20 21 22 23 24 Number of item related to purpose and operation of ongoing BRFSS
Fig. 1. Distribution of experts' score by nine point Likert scale regarding the items related to purpose and operation of ongoing behavioral risk factor surveillance system development in the Republic of Moldova.
*Abbreviations: BRFSS, Behavioral Risk Factor Surveillance System
The contents of core questions related to purpose and operation of ongoing behavioral risk factor surveillance according to their number were, as following: Purpose of ongoing BRFSS
Health-related risk factor events under surveillance:
In terms of providing a complex surveillance of risk factors, the system is required to take into account the following events:
1. Behavioral risk factors: smoking status, healthy diet, alcohol consumption, and physical activity.
2. Biological risk factors: blood pressure, total serum cholesterol, fasting blood glucose, and body mass index.
3. Ongoing surveillance of modifiable risk factors is required along with monitoring of individual coexisting morbidities.
i (
Surveillance indicators:
Calculation of core monitoring indicators in terms of:
4. Events related to monitoring indicators.
5. Quality related indicators of system performance (Response rates).
6. Indicators must respect international common standards.
Level of integration:
In terms of providing an integrated surveillance of risk factors, the system is required to take into account the following:
7. Using the standards of existing ongoing BRFSS, which permits comparability among surveys at the international level.
8. Ongoing surveillance of modifiable risk factors is required along with monitoring of mortality (inclusively premature deaths) and morbidity, all being integrated as an informational system.
9. A systematic and ongoing approach in the surveillance of risk factors helps to provide useful information for the continual monitoring of public health trends.
Operation of ongoing BRFSS Population under surveillance:
In terms of premature deaths, assessment is more logical and efficient:
10. The study population should consist of persons aged between 18 and 69 years old rather than the study population consisting of persons aged 18+ years old.
In terms of mitigating falling survey resolution and response rate, it is more logical and efficient to:
11. Use an approach to select respondents enrolled in the lists of local health units vital statistics, updated on 1st January of the year of the survey than to use the Random Digit Dialing (RDD) approach to select the phone numbers by randomly selecting from commercially available lists of telephone numbers.
Period of data collection:
In order to mitigate falling survey response rates, it is more logical and efficient to do:
12. Sampling based on a monthly collection switched to a four month reporting frame which may provide more time to contact respondents for a refusal conversation
Data collection:
In order to mitigate falling survey response rates, it is more logical and efficient:
13. To provide telephone interview data collection by a Local Health Unit (LHU) working team comprised of coordinators (family doctor, statistician), interviewers (nurses), and public health professionals.
14. To provide telephone interview data collection by a Local Public Health Unit (LPHU) working team comprised of several public health professionals and statisticians.
15. For each unit the monthly, and respectively, annual minimum sample size of completed interview is defined at the beginning of year.
Data management:
16. National coordinating group supervises the application of methods and standards of the process.
55)) -
17. National technical group supervises process of verification and correction of the interviews and makes available dataset after prior quality controls are applied to programs.
18. Regional coordinating group gives assistance and support to the local working teams.
Funding sources:
19. New implementation of BRFSS using existing Health System organization and management provide affordable expenditures.
Personnel requirements:
20. Trained staff at the local level conducts telephone survey.
21. Technical and methodological assistance for training is provided by a coordinating group at the National level.
22. Local working group is composed of coordinators, interviewers, and public health experts.
23. Coordinators are a medical doctor and a statistician already working in local health unit.
24. Interviewers are nurses already working in local health unit.
Consensus of the Delphi survey experts in terms of attributes (14 questions) of the ongoing surveillance system
found potential implementation in the Republic of Moldova
as falling within 7-9 median score (fig. 2).
Fig. 2. Distribution of experts' score by nine point Likert-scale regarding the items related to attributes of ongoing behavioral risk factor surveillance system development in the Republic of Moldova.
"Abbreviations: BRFSS, Behavioral Risk Factor Surveillance System.
The contents of core questions related to attributes of ongoing behavioral risk factor surveillance according their number were as following: Simplicity:
1. Flow chart is structured according to three distinct levels: national, regional and local.
2. To determine standards for system operation and assessing the performance.
Flexibility:
3. Along with a fixed core component of the questionnaire, it is required to provide rotating core components (sets of questions, asked in alternating years by all participating Local Health Units that address different topics).
4. Along with fixed core components of the questionnaire, it is required to provide optional modules (questions that regions select in their questionnaires to achieve needed information).
5. Along with fixed core component of the questionnaire, it is required to provide emerging modules (a few questions for brief periods of time).
6. Only fixed core components of questionnaire are required to be provided during the implementation period (sets of standardized questions, asked in different periods of time by all participating local health units).
Data quality:
7. To utilize the Nationwide standardized structured questionnaire and the surveillance indicators calculation.
8. To share a common information system and database.
9. To provide central supervision and support for training, data analyses and communications activities.
Acceptability:
10. Extracting the sample from local health units' lists of residents vs. extracting the sample from random digit dialing will improve respondent's accessibility.
11. Data collection provided by local health units versus data collection provided by public health units will improve respondent's cooperation.
Sensitivity:
12. Health literacy level improves the ability of persons to understand the questions and correctly identify their status.
13. Continued information about the surveillance system provided not only on regular medical check-up visits but also through the media will improve the willingness of respondents to report their status.
Timelines:
14. Electronic data collection from reporting sources including entry data by web-based system (local/regional/ national access).
When comparing the opinion of international and national experts for every 38 items from core part of the first round questionnaire in 94.7% of cases the significant difference was not observed (p > .05). The opinion of international and national experts was significantly different in case of two questions: one related to surveillance system purpose (item number 6; p=.033) and other related to surveillance system operation (item number 10; p=.019).
The dropout rate for the second round was 21.1% with 15 experts participating (7 international and 8 national). Due to the first round Delphi survey achieved experts' consensus in all items included for discussions, the second round questionnaire was composed of a few questions only to gather experts' suggestions and recommendations in order to provide some specific additional details for reached consensus before.
Discussion
In the literature the optimal number of subjects to involve in the Delphi study is not really defined [15]. However, when the training of experts is similar, ten to fifteen Delphi
y 1 J
— —~ 1
1 2 3 4 5 6 7 8 9 10 11 12 13 14 Number of item related to atributes of ongoing BRFSS
panel participants are sufficient for useful results, even found between five and ten members are reasonable [16,17].
Following the recommendation of previous Delphi study about the importance of high homogeneity of the experts' panel regarding the similar training and general understanding in the field of interest, this study reached the opinions' consensus in the first round of the Delphi written survey with the relatively reasonable numbers (19) of participants. This unexpected rapid consensus achievement could be explained by homogeneity of Delphi panel considered as study strengths as well, along with anonymity and joint but individual thinking. Although however it should be noted that Delphi participants from countries developing ongoing BRFSS based on U.S. standards were underrepresented, as their participation agreement has not been obtained.
This study found that national and international experts highly trained and competent in the area of knowledge related to the target issue mainly expressed (94.7%) the similar opinion. This finding is particularly important to highlight that expectations and needs of ongoing system potential implementation in the Republic Moldova are similarly directed to the recommendations and suggestions of functional and successful ongoing surveillance system in the world, being a great opportunity for feasible development of ongoing BRFSS in the Republic of Moldova.
Limitations of this study are: confusion around the Delphi sample size, non-probability method of sampling, self-reporting data and mainly qualitative approach of the research.
Conclusions
High homogeneity of Delphi panel related to similar trained and competent experts allows for effective and reasonable survey management in terms of achieving feasible consensus agreement.
Acknowledgments
The authors are very grateful for the participation and contribution in Delphi survey, regarding the Development of ongoing behavioral risk factor surveillance in the Republic of Moldova, of national and international experts from the School of Health Professions, EVMS (USA); College of Public Health, The University of Iowa (USA); Division of Public Health Sciences, Department of Social Sciences and Health Policy, Wake Forest University, School of Medicine (USA); School of Medicine & Health Sciences, The George Washington University (USA); National Institute of Health (Italy), Department of Public Health and Management, Carol Davila University of Medicine and Pharmacy (Bucharest, Romania); Lucian Blaga University of Sibiu, National Institute of Public Health (Romania); Nicolae Testemitanu University of Medicine and Pharmacy (Republic of Moldova); Department of Monitoring and Evaluation of Health Policy, Ministry of Health of the Republic of Moldova; National Center of Public Health (Republic of Moldova).
References
1. Centers for Disease Control and Prevention. Behavioral Risk Factor
Surveillance System. https://www.cdc.gov/brfss/. Accessed July 29, 2017.
i 157
2. World Health Organization. Global Status Report on Noncommunicable Diseases, 2014.
3. World Health Organization. Noncommunicable diseases (NCDs) country profiles: Republic of Moldova, 2014.
4. Day J and Bobeva M. A generic toolkit for the successful management of Delphi studies. The Electronic Journal of Business Research Methodology. 2005; 3(2):103-16. http://ejbrm.com/volume3/issue2/p103. Accessed June 15, 2017.
5. Hotararea Parlamentului Republicii Moldova nr.82 din 12.04.2012 penru aprobarea Strategiei nationale de prevenire si control al bolilor netrans-misibile pe anii 2012-2020 [The Republic of Moldova Parliament Decision no.82 of 12.04.2012 regarding National Strategy on noncommunicable diseases prevention and control for the period 2012-2020]. Monitorul Oficial Nr. 126-129/412 din 22.06.2012.
6. World Health Organization. Global Action Plan for the Prevention and Control of NCDs 2013-2020. Geneva, 2013.
7. Hotararea Guvernului Republicii Moldova nr.300 din 24.04.2014 cu privire la aprobarea Programului national de prevenire control al bolilor cardio-vasculare pentru anii 2014-2020 [The Republic of Moldova Government Decision no.300 of 24.04. 2014 regarding the approval of the National Program on cardiovascular diseases prevention and control for the period 2014-2020]. Monitorul Oficial nr.104-109/327 din 06.05. 2014.
8. Hotararea Guvernului Republicii Moldova nr.403 din 06.04.2016 pentru aprobarea Planului national de actiuni pentru anii 2016-2020 privind implementarea Strategiei nationale de prevenire control al bolilor ne-transmisibile pe anii 2012-2020 [The Republic of Moldova Government Decision no.403 of 06.04.2016 in terms of the approval of the Action Plan for the period 2016-2020 regarding the implementation of National Strategy on noncommunicable diseases prevention and control 2012-2020]. Monitorul Oficial nr.100-105/464 din 15.04. 2016.
9. Hotararea Guvernului Republicii Moldova nr.300 din 24.04.2014 cu pri-vire la aprobarea pentru aprobarea Planului de actiuni pe anii 2014-2020 privind implementarea Programului national de prevenire control al bolilor cardiovasculare pentru anii 2014-2020 [The Republic of Moldova Government Decision no.300 of 24.04. 2014 regarding the approval of the Action Plan for the period 2014-2020 regarding the implementation of National Program on cardiovascular diseases prevention and control for the period 2014-2020]. Monitorul Oficial nr.104-109/327 din 06.05. 2014.
10. Ababii I, Vataman E, Raevschi E, Pautz L. Pilot cross-sectional telephone survey test mainly based on the U.S. BRFSS's protocol conducted in the Republic of Moldova: challenges of the surveillance quality. International Journal of Humanities and Social Sciences. 2016; 5(6): 73-80.
11. Pautz L, Raevschi E, Patel A, Ciubotaru E. Development of Behavioral Risk Factor Surveillance System Management in the Republic of Moldova based on the U.S. BRFSS Standards. International Journal of Business and Management Invention. 2016; 5 (7): 33-36.
12. Raevschi Elena. Supravegherea factorilor de risc responsabili de dezvolta-rea bolilor cardiovasculare in Republica Moldova [Surveillance of the risk factors responsible for the development of cardiovascular diseases in the Republic of Moldova]. Buletinul Academiei de Stiinte a Moldovei. Stiinte Medicale. 2016; 51 (2): 97-104.
13. Penina OA, Raevski EM. Otsenka vozrastno-prichinnykh grupp riska izbytoch'noy smertnosti v Respublike Moldova [Assessment of age-related risk groups for excess mortality in the Republic of Moldova]. Byuleten natsional'nogo nauch'no issledovatelskogo instituta obschestevennogo zdorovia imeni N.A. Semashko [Bulletin of Semashko National Research Institute of Public Health]. Moskva, 2017; 1:220-225.
14. Centers of Disease Control and Prevention. Updated Guidelines for Evaluating Public Surveillance Systems. MMWR Recommendations and Reports 2001; 50(RR13): 1-35. Atlanta,2001.
15. Ralita B Akins, Homer Tolson and Bryan R Cole. Stability of response characteristics of a Delphi panel: application of bootstrap data expansion. BMC Medical Research Methodology. 2005; 5(37). Doi:10.1186/1471-2288-5-37.
16. Ziglio E. The Delphi method and its contribution to decision-making. In: Adler M. and Ziglio E.(Eds). Gazing into the Oracle the Delphi method and its application to social policy and public health. London, 1996; Jessica Kingsley Publisher, 3-33.
17. Hsu Chia-Chein and Stanford B. The Delphi technique: making sense of consensus. Practical Assessment Research & Evaluation. 2007; 12(10):1-10.
