Biomedical Research

Abstract

Objective: This thesis was designed to explore the application of Delphi method in the design of hospital pharmacy administration evaluation index system, and to establish a scientific method of the evaluation index system for hospital pharmacy administration quality.

Method: The Delphi method was taken to summarize the opinions as the reference to solicit expert opinions by correspondence, and finally a relatively consistent opinion would be obtained.

Results: Through two rounds of consultations, the drug management evaluation index system was divided into 2 levels of index, and the first level consists of 3 indexes, respectively including the drug quality assurance system, the rational drug use evaluation system and the pharmaceutical service implementation system; whereas the second level were composed of several indexes under the first level.

Conclusions: The Delphi method is a systematic, comprehensive, reliable and objective method, which can be applied to design the hospital pharmacy administration evaluation index system.

Keywords

Delphi, Pharmacy administration.

Introduction

Since the Ministry of Health and State Administration of Traditional Chinese Medicine of the People’s Republic of China promulgated Provisional Regulations on Administration of Pharmaceutical Affair in Medical Institutions on January 21, 2002, in order to adapt to the development of pharmacy administration with clinical pharmacy as its core and promote the pharmaceutical technology services of clinical rational drug use and related drug administration work [1], hospitals in the second tier and above have set up pharmacy administration organizations such as the pharmacy administration commission, and other medical institutions have also established the pharmacy administration group. However, over a long time, there has been little or no index that can objectively evaluate performance of the organization. Therefore, there is an urgent need to scientifically, systematically, effectively and comprehensively evaluate the quality of hospital pharmacy administration and design the related evaluation index system for guiding and promoting the hospital pharmacy administration.

The Delphi method is a quantitative and qualitative forecasting and evaluation method that expert opinions are extensively solicited in an anonymous way, such opinions are converged towards a consistent opinion after repeated information exchange and feedback correction, and the evaluation object is finally evaluated.

In this method, the researcher prepared the questionnaire and consult the members of expert group by correspondence in accordance with established procedures; and the members of expert group submitted their views anonymously (by correspondence). After several repeated inquiries and feedback, the members of the expert group gradually converged on their views, and finally the collective judgment results were obtained with a very high accuracy. At present, the Delphi method is widely used in the evaluation of hospital drug management risks and Traditional Chinese Medicine (TCM) syndrome, yet it has not been applied to the design of hospital pharmacy administration evaluation index system [2-10]. Therefore, this thesis aims to explore the application of the Delphi method in the design of hospital pharmacy administration evaluation index system and to establish a scientific method of the evaluation index system for hospital pharmacy administration quality.

Objects and Methods

Objects

The selection of experts determines the authority, science and effectiveness of the conclusion, and it should be carried out in strict accordance with the following conditions: The expert shall work in the health, medicine and management for more than 10 y, get familiar with the hospital pharmacy administration and assessment process, and possess rich practical experience. A panel of experts shall be selected according to the required scope of knowledge. Besides, the panel of experts covers varied fields, such as health, drug supervision, hospital and research. Based on the project size and coverage, 15-20 experts may be selected.

Delphi method

The Delphi method is a management technique that a one-way anonymous method (correspondence) is used to solicit opinions in accordance with the predefined system procedures, and the members of the panel do not discuss and contact with each other but contact with the investigators, in order to reach an ultimate consensus through repeated consultation, get a satisfactory and scientific program, and tackle various complex problems. Through repeated consultation and feedback, a collective judgment with high accuracy will be obtained [11].

Specific implementation

The implementation process for the Delphi method consists of four rounds. In the first round, the facilitator sends out the open questionnaire to experts, in which only forecasting questions are proposed for experts to answer. Subsequently, the facilitator provides a summary of the experts' questionnaire, develops a list of forecasts by using accurate terminology, and designs a second questionnaire for experts. in the second round, the questionnaire requires the panel to evaluate the experts’ forecast, and then the facilitator provides a summary of expert opinions in the second round, makes statistics, and designs a third questionnaire for experts. The panelist comments on arguments by reviewing them, evaluates conflicting opinions, and gives reasons for changing previously stated opinions. At this time, the facilitator collects new opinions provided by the panelist, makes statistics of them together with the summary in the second round, and designs a fourth questionnaire for experts. New forecasts will be given by the panelist through reevaluation, analysis and weighing. The forecasts will be submitted, summarized and statistically processed.

According to different situations, a unanimous conclusion can be drawn by two rounds and it is unnecessary to proceed the third and fourth rounds; some events end in the fourth round, yet a unanimous conclusion cannot be drawn. Among these, the minimum standard shall be quantified and specified towards the classification of the evaluation grade, judgment criteria, and degree of familiarity [12]. According to assignment by 5-Point Likert scale and distinct expert authorities over questions because of individual differences, the specific criteria determined are listed in Table 1.

Table 1. Table of quantified evaluation grade, judgment criteria and degree of familiarity.

For specific research technique route of the Delphi method in the design of hospital pharmacy administration evaluation index system. Basic principles determined by the index weight are: 1. Principle of Integrity, i.e., each index shall be placed in the hospital pharmacy administration institution to be evaluated to determine its importance in the overall operation; 2. Materiality principle, i.e., when the overall situation is reflected, the relative importance index has high relative importance weight, and otherwise low; 3. Principle of relativity, , i.e., the weight of an index might have multiple values in different contents, and each weight corresponds to the determined content (Figure 1).

Figure 1: Research technical route of the Delphi method in the design of hospital pharmacy administration evaluation index system.

Expert positive coefficient and screening criteria

Expert positive coefficient: Expert positive coefficient indicates the concern level of the interviewed experts in the study, expressed as the questionnaire collection rate: Questionnaire collection rate=number of questionnaire collected/number of questionnaires sending × 100%.

Index screening criteria: the index’s importance assignment mean is greater than 4.00 and the variation coefficient is less than 0.22, and the index screening result is finally determined by combining expert opinions with collective opinions of the research group.

Statistical processing

The expert positive coefficient is expressed by the effective response rate of questionnaire, i.e., the number of valid questionnaire is divided by the total number of the questionnaires sending; the index’s importance assignment mean is used to express the convergence of expert opinions; the statistical processing of all the data is done by using SPSS 18.0 statistical software.

Results

Basic information and authority level of experts

In this study, 21 experts were invited. Experts come from a wide range of fields, covering the health care, medicine, management and other related fields. The average age of experts was (57 ± 6.68 y), and the average seniority was (28 ± 7.53 y, Table 2). Two rounds of consultation were conducted. in the first round, 21 questionnaires were sent out and all were collected; there were 20 valid questionnaires, and the effective response rate of questionnaire was 95.2%; the index judgment coefficient of experts was 0.92; experts’ degree of familiarity with index was 0.83, i.e., the authority level of experts was 0.89. In the second round, the questionnaires were sent to experts who provided valid responses in the previous round, 18 questionnaires were collected, and all were valid. The valid response rate was 85.7%; the index judgment coefficient of experts was 0.93, and experts’ degree of familiarity with index was 0.79, i.e., the authority level of experts was 0.86.

Table 2. Basic information of experts.

Structural analysis of drug management evaluation index system

The convergence of expert opinions is mainly expressed with importance score. After two rounds of consultation, the expert opinions show good convergence. In the first round of consultation, the average importance score for all indexes was 4.87; the average importance score for all indexes in the second round was 4.28. The variation coefficient reflects the volatility of the expert's evaluation on the index importance. Among all indexes, the minimum and maximum variation coefficients were 0.015 and 0.217 respectively, i.e., it was considered that expert opinions meet the required convergence at this time, and the weight distribution was reasonable. Among them, after two rounds of screening, the drug management evaluation index system was determined: the indexes were divided into two levels. There were 3 indexes in the first level, including the drug quality assurance system, rational drug use evaluation system, and pharmaceutical service implementation system, whereas the second level were composed of several indexes under the first level. The detailed evaluation structure is shown in Table 3.

Table 3. Index evaluation structure.

Discussion

A scientific, reasonable and fair hospital pharmacy administration evaluation index system not only directly affects the hospital pharmacy administration work, but also affects the people's drug use safety and even the development of the entire medical health services. At present, the establishment of the hospital pharmacy administration index system is still in the exploratory stage, and there is no available model, so it is necessary to obtain a wide range of opinions of experts from relevant fields.

The Delphi method is a quantitative and qualitative forecasting and evaluation method that expert opinions are extensively solicited in an anonymous way, such opinions are converged towards a consistent opinion after repeated information exchange and feedback correction, and the evaluation object is finally evaluated [13]. It is essentially a group decision-making based on theories and practical experience. In the absence of historical data, the Delphi method has become the most important method, which is applied to various fields, especially to the establishment and determination process of various index systems. Currently, there is no authoritative, recognized and unified evaluation index system in the hospital pharmacy administration, and drug quality, drug market supervision and clinical use of drugs and other problems are recurring. As a result, it is urgent to evaluate the quality of hospital pharmacy administration in a scientific, objective and effective way and promote the hospital drug management. It shows that the Delphi method may well make up the deficiencies of current pharmacy administration.

In this study, the Delphi method is applied to design the hospital pharmacy administration evaluation index system of two levels of indexes, which can reflect three problems affecting the quality of hospital pharmacy administration in a scientific, objective, true, comprehensive and effective way. These are qualitative indexes that reflect problems from different aspects, and there is no comparability between each other, so it is difficult to determine their relative importance. The proposed Delphi method is a relatively effective way to deal with complex problems. It facilitates to achieve more scientific and reasonable results by combining repeated solicitation of expert opinions with scientific statistics.

By adopting the principle of paying equal attention to representative and authoritative considerations and combining academic experts with management experts, the researcher selects technical personnel and officials of relevant government departments related who engage in the administration of key medical disciplines as well as the academic leaders who specialize in key disciplines; professionally, there are administrators who engage in key medical disciplines and expert professors who are involved in clinical work; geographically, there are professors and experts from provincial medical and health institutions, as well as professors and experts from universities and other cities and municipal medical institutions. All these experts are highly representative, ensuring the reliability of consultation findings.

Inevitably, any method has flaws. When the Delphi method is applied to the design of hospital pharmacy administration evaluation index system, its limitations are revealed. Limitations are mainly [2] as follows: 1. It needs longer time; 2. it is difficult to avoid subjective factors; 3. some skills are needed for organization and leadership. in addition, the established second-level indexes are not segmented. Threelevel indexes might be developed under more second-level indexes; hospitals of different classes or natures are not considered, which should be distinguished by the evaluation index system. However, it does not affect the science, reasonableness and integrity of the findings. Such indexes can be further segmented by using the Delphi method in the future research.

As medical technology continues to develop and there are increasingly high demands for medical services, the hospital pharmacy administration also shifts from the guaranteed supply to the technical service management, and transforms from the “drug-oriented” management of traditional drug procurement, storage, distribution, quality and economic management to “people-oriented” management; i.e., it develops into a systematic pharmacy administration with rational drug use over patients as the core [14]. Therefore, the importance of original indexes may be changed, followed by some new and better indexes. As a result, the hospital pharmacy administration evaluation index system should also be closely integrated with actual conditions for constant improvement and optimization, so that the hospital pharmacy administration institutions can better work and serve patients.

References

1. Jingbo T. International network communications of rational drug use. Chinese Yearbook in 2000~2001. Beijing China Sci Technol Press 2001; 38-71.
2. Jing W, Dao Z, Xinmin H. Delphi method and its application in traditional Chinese medicine syndrome. Shanxi J Trad Chinese Med 2007; 23: 75-77.
3. Lei Z, Hongyan B, Lina X. Integrated application of the Delphi, risk matrix and borda count methods in assessment of risks of drugs management in wards. Clin Med J 2015; 13: 29-33.
4. Long Y, Chen Y. Application of Delphi method in the design of evaluation indux system of hospital pharmacy administration. Chin Hosp Pharm J 2017; 37: 181-184.
5. Wang G, Jiang X, Zhang Y. Eatablishment of an evaluation index system for health literacy among chronic disease patients with Delphi method. Chinese Public Health 2017; 36: 18-22.
6. Nath CR, Sylvester ST, Yasek V, Gunel E. Development and validation of a literacy assessment tool for persons with diabetes. Diabetes Educ 2001; 27: 857-864.
7. Yang Y. Construction of public hospital reputation evaluation index system based on Delphi and AHP. Chinese Health Serv Manag 2017; 4: 241-283.
8. Brown P, Panattoni L, Cameron L. Hospital sector choice and support for public hospital care in New Zealand: Results from a labeled discrete choice survey. J Health Econom 2015; 43: 118-127.
9. Bouphan P, Apipalakul C, Ngang TK. Factors affecting public health performance evaluation of sub-district health promoting hospital directors. Proc Soc Behav Sci 2015; 185: 128-132.
10. Ko M, Derose KP, Needleman J. Whose social capital matters? The case of U.S. urban public hospital closures and conversions to private ownership. Soc Sci Med 2014; 114: 188-196.
11. Hongyan Y, Xiaoyun W, Guangning N. Application of Delphi method in menopausal syndrome rating scale index screening. J South Med Univ 2007; 27: 562-564.
12. Guang Z. Modern epidemiological methods and its applications. Peking University Health Science Center and Peking Union Medical College Press, Beijing, China 1994; 215.
13. Lifeng T, Dongping H, Xiling S. The application of delphi method in evaluating the possibility and consequence of public health emergencies in regional large-scale activities. Occup Health 2012; 28: 1160-1164.
14. Runqing C. Some thoughts on pharmacy administration of primary-level hospital in China. Guangdong Pharm J 2003; 13: 58.