Next Article in Journal
Have the Reports of TQM’s Death Been Greatly Exaggerated? A Re-Examination of the Concept’s Historical Popularity Trajectory
Next Article in Special Issue
The Issues and Challenges of Local Government Units in the Era of Population Ageing
Previous Article in Journal
Embracing Variety in Decision-Making Regarding Adoption of Industry 4.0
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Indicators of Efficiency in the Pharmaceutical Management of a Public Health System

by
Manuel Martínez-Blanes
1,*,†,
Francisco Javier Rondán-Cataluña
2,† and
José Manuel Velarde-Gestera
3,†
1
Departamento de Economía, Métodos Cuantitativos e Historia Económica, Universidad Pablo de Olavide, Ctra. Utrera (A-376) km 1, 41013 Seville, Spain
2
Dpto. Administración de Empresas y Marketing, Universidad de Sevilla, Ramon y Cajal, 1, 41018 Seville, Spain
3
Drimay Consultores, Alberche 4A -1º, 41005 Seville, Spain
*
Author to whom correspondence should be addressed.
These authors contributed equally to this work.
Adm. Sci. 2020, 10(2), 31; https://doi.org/10.3390/admsci10020031
Submission received: 23 March 2020 / Revised: 13 May 2020 / Accepted: 14 May 2020 / Published: 19 May 2020
(This article belongs to the Special Issue Evidence-Based Public Policy Making)

Abstract

:
This practical research arises from a growing interest in offering models that provide a measure of the efficiency of pharmaceutical management in the Public Health System, beyond spending per inhabitant or other similar indices that do not incorporate the effects of the environment. The main objective of the paper is to design a tool that can measure the relative efficiency of health systems, with respect to the rational use of medicament based on its regional socioeconomic context. The first step was to check if it is possible to apply the efficiency and productivity analysis models, widely used in the economy and companies. We have carefully chosen the context factors that are pertinent and influential in the final index: demographic, sanitary, economic, and social. After selecting and ordering the context variables of the different regions of Spain, they are normalized using the index number transformation or ‘distance’ to a reference. The weighted sum method is used to build the synthetic indicators. The main contribution of this paper is to propose a relative efficiency indicator that assumes the context of performance and allows a comparison between health organizations. The methodology offered in this manuscript could assist policymakers to make better decisions in order to enhance the productivity of the public pharmaceutical system, and it makes available feedback about past decisions.

1. Introduction

At a time when the epidemiological crisis of the coronavirus is putting the health systems of all the countries of the world at the razor edge, analyzing the efficiency of the pharmaceutical public systems becomes a fundamental aspect for health managers and politicians. The utilization of medications carries on rising in the Organization for Economic Co-operation and Development (OECD) countries and forcing up pharmaceutical spending. However, cost-containment plans and patent expirations of some top-selling products have put a downwards pressure on pharmaceutical payments. Furthermore, the spread of high-cost specialty drugs will be the primary handler of health spending progression in the next years (Belloni et al. 2016). Evaluating the efficiency of health care systems concerning the frontier specified by a comparable sample of countries or regions could be helpful in identifying whether there is a potential correction in the use of resources (Varabyova and Müller 2016).
About concrete characteristics of the pharmaceutical industry, such as research and development intensity, patents, the uncertainty of the product development process, lack of performance measurement, and the design of indicators to evaluate it, is becoming more and more relevant (Shabaninejad et al. 2014; Roemer-Mahler 2013).
The main objective of the paper is to design a tool that is capable of measuring the relative efficiency of health systems, with respect to the rational use of medicament based on its regional socioeconomic context. This study could contribute to analyze the socioeconomic inequalities in health in Spain due to the different regional health policies, a fact that is manifested in previous studies (López et al. 2019; Méndez-Castrillón Susín 2014). This circumstance is present in other decentralized countries such as the USA, Germany, Belgium, Australia, Mexico, Canada, etc.
Health services are subjected to state regulation and minimal regulatory margins; we cannot assume that the primary resources to obtain the final product (the catalog and regulated price of medicines, therapeutic prescription guidelines, the training of doctors, etc.) are similar for all health services. Although there is some space for ideas and own initiatives that mark differences in the management of health services, we do appreciate an external influence on the efficiency of public pharmaceutical management that does not exist in most of the other industries: the goal client cannot be chosen. Definitively, a regional health service, cannot choose the geographical, demographic, social, and economic environment where it works. We have called these conditions the external context.
Once this influence that limits management and biases performance is accepted, it is essential to use tools that allow measuring efficiency, extracting it from the context.
Given these issues and the multidimensionality of the aspects that impact public pharmaceutical management (Nardo and Saisana 2008), we have designed an impartial synthetic indicator to evaluate the competence of public pharmaceutical management.

2. Measuring the Efficiency of Public Pharmaceutical Management

Measuring the efficiency of some process of a health organization is an arduous and challenging task due to the peculiarities of the market and environment where it operates. This task becomes even more complicated if the object of study is public health, where the differences concerning the most common models studied by econometric theories increase (Torchia et al. 2015). Some peculiar features are (Méndez-Castrillón Susín 2014):
  • There is no freedom to choose the clients or the area of action.
  • The competence of other similar public organizations in the geographical area of action is very limited or non-existent.
  • The main objectives are oriented towards quality (health outcomes: life expectancy, years of life adjusted by quality, etc.) and the access of everyone to the services; and not so much towards efficiency. The measurement of organizational efficiency is only one of the variables of the health production function.
Public healthcare managers are more worried about some persistent concerns, such as waiting times for consultations and care, lack of patient-centeredness, and health services are sometimes distributed unfairly (Anell 2015). For example, some recent studies have found significant alterations in waiting times in public hospitals in the United Kingdom, involving patients with different socioeconomic positions (Moscelli et al. 2015). The problems of different public health services in several regions inside a country are typical in Spain, where decision making in health care is decentralized (17 regional governments). However, capital for health care is an increased expenditure in most developed countries and regions due mainly to technological development and the aging of inhabitants (Simonet 2015). As a result, the efficiency of healthcare expenditures in public organizations is becoming more and more relevant. Besides, health organizations often pursue multiple objectives, and their production processes are difficult to standardize. Consequently, the analysis of the efficiency of their processes is complicated.
Economic efficiency aims to measure the relationship between an output and an input, between a product and resources to put it into effect (Parra Rodríguez 2011). A system is efficient when it maximizes the desired outputs given available inputs (Farrell 1957). Additionally, the efficiency of a system can be measured by final products or by intermediate products. Health is a paradigmatic case of the existence of an essential difference between the intermediate product and the final product. The final product is the contribution of health services to improve the health status of individuals. In general, empirical studies measure health services through activity measures (intermediate products). The selection of the representative variables of the service and the resources always implies the adoption of various assumptions about the quality of the service, the adequacy of the attention, and the seriousness of the patients attended. This work focuses on a barely studied intermediate product: pharmaceutical management in public healthcare, which is influenced by general health policies and needs its objectives.
The purpose of public pharmaceutical management is to ensure that patients receive the medication appropriate to their needs, based on the best scientific evidence and the highest quality; with the appropriate benefit/risk ratio; in doses corresponding to their requirements, and at the lowest cost to them and the community (Abbott 2005). It meets the goals of public health: universal access and quality and, lastly, it incorporates efficiency. Policymakers’ attention in socioeconomic discrimination in health also spreads further than measurement, through to clarifying and comprehending its motivating roots (Heckley et al. 2016). This objective defines, in turn, the final aim of pharmaceutical management: the necessary and appropriate medication at the lowest cost.
Efficiency measurement studies can be classified in frontier analysis and non-frontier analysis (Mutis 2006), depending on the explicit construction of an efficiency frontier. Non-frontier analysis, developed in the conceptual framework of health and epidemiological management, focuses on obtaining indicators of specific dimensions relevant to policymakers and health managers (costs, productivity, quality, etc.). This analysis measures efficiency from groups of partial indicators. Although it lacks the formal rigor and parsimony of the border approach, it presents a greater wealth of information, and it enables comparisons between healthcare organizations of specific dimensions (Martín and López del Amo 2007).
Frontier analysis bases its methodological strategy on the explicit construction of an efficiency border in the configuration of a model with which to compare it. This means that it should be precise and general enough so that it can be used as a gold standard for the largest number of institutions. This is one of the goals of this analysis. Frontier methods are mostly classified into nonparametric methods, such as Data Envelopment Analysis (DEA) and Free Disposal Hull (FDH), and parametric methods, such as Stochastic Frontier Analysis (SFA) (Varabyova and Müller 2016; Coelli et al. 2005).
The frontier methodology is frequently used to measure the efficiency of public organizations, and it is known as the hidden cost of public production. This consists of comparing the cost incurred by a Public Health Service in producing a particular good or service with the cost of production that would be obtained if the good or service was carried out by a private company, the latter using the same production factors as the former. The idea is to know the excess or deficiency of the cost incurred by the public sector concerning the private sector, and this excess or deficiency is attributed to the lesser (inefficiency) or greater efficiency of the Administration (Wagstaff 1989; Afonso and Kazemi 2017). In our case, this method is revealed to be inadequate because private health companies do not usually fund medication, only that provided in hospitals. Therefore, the frontier efficiency model must be sought in the public sector. Considering that a commonly accepted reference is the national average for indicators such as the growth of spending or spending per capita, we have built our efficiency frontier around this national average. The frontier variable that we have chosen is the pharmaceutical spending per inhabitant that reflects the objective of pharmaceutical management efficiency: provide the right medication at the lowest cost.
With public health services subject to state or regional regulations and with some minimal regulatory margins, the assumption can be made that the primary resources to obtain the final product (the catalog and regulated price of medicines, therapeutic prescription guidelines, the training of doctors, etc.) are similar at some extent for all public health services. Although there is some space for ideas and own initiatives that mark differences in the management of health services. However, we do appreciate an external influence on the efficiency of public pharmaceutical management that does not exist in most of the classic examples: the goal client cannot be chosen. Definitively, a regional health service cannot choose the geographical, demographic, social, and economic environment where it works. We have called these conditions the external context.
Once this influence that limits management and biases performance is accepted, it is vital to use tools that allow measuring efficiency, extracting it from the context.
Taking into account these considerations and the multidimensionality of the factors that influence public pharmaceutical management (Nardo and Saisana 2008), we have created an objective synthetic indicator to measure the efficiency of public pharmaceutical management.

3. Materials and Methods

In this manuscript, we study the relative efficiency in the management of pharmaceutical expenditure by comparing it with a frontier indicator. A context indicator was chosen, following the guidelines of the European Union (European Union 2018), as it provides a simple and reliable base to describe a context variable that allows highlighting the most critical points of the regional context.
The following chart shows the methodological scheme followed by the authors (Figure 1):
A synthetic indicator has the virtue of summarizing complex situations, simplifying an analysis. In order to achieve maximum credibility, the choice of the variables and the formula of their combination must be well justified and explained so that it can be discussed and verified by other researchers and scholars.
For the right choice of indicators that facilitate control and management, these must meet the following characteristics (CEPYME 2011):
  • Objectives: a standard methodology should provide data available for all the services to compare.
  • Precise: if possible, quantitative, and not qualitative.
  • Accessible: other researchers can verify them.
In our case, we have first selected the context factors that are relevant and influential in the final product. We have considered four: demographic, sanitary, economic, and social. All this information are secondary data for each region obtained from the National Institute of Statistics of Spain.
For each of them, and in order to carry out a practical demonstration of the application, we have selected the indicators that, according to the literature (Wang et al. 2019; Häkkinen and Joumard 2007; Domínguez Serrano et al. 2011), better represent each factor. Given that this work aims to define a new method supported by a tool to be used by managers, it will be the experts in the specific field of application who agree on the number of factors, their representatives, and their weight when constructing the synthetic index.
We have carried out a correlation analysis between the factors and the border indicators, and the highest correlation is obtained with the factors age and frequency. Frequency is the number of medical visits to primary care centers.
After selecting and ordering the context variables of the different regions, they are normalized using the index number transformation or ‘distance’ to a reference (Mazziotta and Pareto 2013). Specifically, all values are transformed into dimensionless units of base 100, with 100 being the value of the national average of the context variable. The indicators can be compared among themselves using this procedure. In the same way, we apply this normalization method to the frontier indicator.
The next step is to assign the weights to the different indicators, according to the degree to which they influence spending. As a type of weighting following to Mazziotta and Pareto (2013), due to the type of final decision maker, for ‘subjective’ weights can be set by a group of policy makers. They will consider the degree to which each factor influences pharmaceutical spending. Initially, we have taken an equal influence for each indicator, assuming 25% of the weight for each of the four indicators because we do not provide more weight to some indicators over the rest.
Among the different possible aggregation procedures, the OECD/JRC Handbook on constructing composite indicators (Nardo and Saisana 2008) focused on the two most used in practice: weighted additive aggregation (WAA) and the weighted product method (WPM).
Due to the type of analysis and the subject to evaluate, we have chosen the WAA, also known as the weighted sum method, which is one of the most used to build synthetic indicators (Nardo and Saisana 2008).
Its formulation is as follows:
I S i = j J w j I i , j   ,
where wj denotes the weight assigned to the jth indicator.
We apply this formula multiplying each of the four indicators by its weight, add them up and then get a joint index for the context of each region. In this way, we get an overview, in a unique value, of the distance that exists between a region and the average of the country. To discuss the different forms of weighting of the components of the synthetic index is not attempted. For this reason, an equal weight has been chosen for each component. The authors understand that the possible use of the tool by policy makers will imply that they decide on a weighting system.
The weighting by agreement involves a risk of partiality and therefore a lack of representativeness. This possible bias can be largely corrected by carrying out a prior statistical analysis measuring the degree of association between the different components of the synthetic index and the border index.
Specifically, an option is to calculate the partial correlation coefficients (Freund et al. 2010) between the aging, chronicity, unemployment, and the number of visits in primary health care and the border index, to see how much variance of the latter is explained by the components of the synthetic index. In this way, the relative importance of each of the components can be determined.
Let Y = b 0 + i = 1 K b i   X i the multiple linear regression model between the variable Y and the predictor variables Xi ,   i = 1 , 2 , , k , and let r y i ( 1 , 2 , n 1 ) the partial correlation coefficient between the variable Y y and the variable   X i knowing that the variables   X j   ,   j = 1 , 2 , , n 1 are already included in the model (see as Appendix A).
We then define the weights of the components of the synthetic index as follows:
ω i = r y i ( 1 , 2 , n 1 ) W   ,   where   W = j = 1 n r y j ( 1 , 2 , n 1 )  
In the annex, we are going to apply this weighting alternative to the variable “pharmaceutical expenditure by inhabitant” to show how the results change.
In the last step, we chose the border indicator: pharmaceutical expenditure per inhabitant. Also, the model has been applied to other border indicators, such as health expenditure per inhabitant and the number of prescriptions per inhabitant. The national average (100) is taken as a base.
At this moment, we already have the two indicators to create an instrument of analysis: the border and the synthetic indicator. The analysis can be represented visually in a scatter chart (Figure 2).
The measure of relative efficiency is the distance to the diagonal that mark the points in which the average expenditure per inhabitant (the chosen border indicator) is equal to the synthetic context indicator. This can be interpreted as the line where spending fits the environment and separates the most efficient regions, located below the diagonal, from the least efficient. The regions should focus their efforts on getting as close as possible to the diagonal of points through pharmaceutical efficiency. Also, when we refer to the average in the previous paragraph, we are talking about the simple average, to which we have not applied the correction factor derived from the context. With this factor of correction, the average is located on the diagonal.
The graph is divided into four quadrants. Quadrant IV, bottom right, is characterized by having unfavorable conditions but spending less than average, a symptom of the proper functioning of its pharmaceutical services. The regions located in this quadrant are in blue. Quadrant I, upper right, is the most sensitive group, formed by the regions that have an unfavorable context and, besides, have an expenditure above average. We have represented them in yellow. In quadrant II, upper left, we obtain the regions where the conditions are favorable and also have an above-average expense; these are in brown. Finally, quadrant III, bottom left, are regions with a favorable context and a reduced expense. These are grey. The central point, represented in black, is the national average.
Now, we describe the indicators that we have used to form the Synthetic context indicator. We have chosen an indicator for each of the four factors considered relevant in public pharmaceutical expenditure.
If we consider first the main causes of pharmaceutical expenditure, there are demographic factors, more specifically the rate of aging. The main factor causing high spending is the percentage of the aged population. This group of people over 64 is not only more at risk of getting ill but also represents a high level of pharmaceutical expenditure.
The index that has been used is:
AGING: (population over 64 years old)/(total population) × 100.
This indicator allows us to establish a relationship between the population resident in each region that exceeds 64 years old concerning the total population of each community, indicating that the volume of the population is already in retirement age, which is the one that creates higher pharmacist spending. These data are obtained from the national statistical institute (INE) for each region.
In 2017, by region, the most favorable percentage of aging stood at 15.49% of the population, while the highest was 24.52%; 18.67% of the Spanish population was elderly (Source: INE and own elaboration).
Secondly, we focus on analyzing what the illness or health burden means for the Spanish health system. According to the WHO (2005), around 60% of the deaths in the world are due to some chronic disease, and a bit less than 50% of the European population suffers some (this proportion reaches 75% if we speak of people over 65 years old). Chronic diseases include heart disease, stroke, cancer, chronic respiratory diseases, and diabetes. Visual impairment and blindness, hearing impairment and deafness, oral diseases, and genetic disorders are other chronic conditions that account for a substantial portion of the global burden of disease.
Special care must be taken with this group of people since they assume approximately 80% of the cost (WHO 2005), and half of them do not follow the treatment correctly.
The index used is:
CHRONICITY: (total number of inhabitants suffering from a long-term illness or health problem)/(total number of inhabitants over 15 years old).
In 2017, Spain had 64.20% chronically ill, 53.64% is the region with the most favorable figure and 79.30% the region with the most worrying number. These data are obtained from the national statistical institute of Spain for each region.
Third, we analyze the economic context of families. A high-income-family can opt for private insurance or go to a pharmacy and buy drugs directly. Double insurance also decreases pressure on public health systems and therefore reduces the possibility of accessing public prescriptions. Two different indicators have been used for this section:
UNEMPLOYMENT RATE: (Number of unemployed)/(active population) × 100.
Unemployed are those who are out of work and who are actively looking for a job, and active population is the total number in the labor force. In 2017, the region with the highest unemployment figure had 26.22%, while the most advantaged region only reached 10.24%. Unemployment in Spain stood at 17.22%. All the information is obtained from the national statistical institute.
INCOME LEVEL: (Average annual income per household).
The region where families have the highest incomes has an average of €34,203 per year, and the poorest is at the level of €20,395 per year. The average of a Spanish family is €27,558 per year. As in the previous indices, the information is extracted from INE, using data from 2017.
Finally, the analysis of the social context intends to present the influence of habit in the use of health systems. We follow the idea that the more frequent the clinical consultation, the higher the cost of healthcare, (Patel et al. 2015), and, therefore, a higher pharmacist cost. It is also interesting to analyze the factors that influence a high or low frequency of consultations since all regions should have similar values (López and López 2013).
For this analysis, the average between the frequency of general medicine and pediatrics is used:
FREQUENCY IN GENERAL MEDICINE: (Number of medical visits to primary care centers)/(population resident in the community).
In 2017, the national average stood at 5.07, while the lowest figure was at 3.77; 6.82 times is the highest number of frequencies in family medicine (Source: MSCBS 2017).
FREQUENCY IN PEDIATRICS: (Number of pediatric visits to primary care centers)/(population resident in the community).
The figure in Spain stands at 4.94 visits, the region with the highest number of visits has 5.97, and the number of the region with the least pediatric visits has 3.78 (Source: MSCBS 2017).
FREQUENCY IN GENERAL MEDICINE AND PEDIATRICS: The average number of visits in medicine and pediatrics in Spain is 5.25. There are 7.92 visits in the region with the highest figure and 3.5 visits in the most active region in this regard (Source: INE 2015).
Also, we have the following border indicators on absolute figures of expenditure that will help us to compare the relative efficiency between regions.
Main border indicator:
PHARMACEUTICAL EXPENDITURE BY INHABITANT: (Total pharmaceutical expenditure)/(resident population in the community).
This allows making comparisons between regions and detecting where the consumption, purchase, or prescription of medications is ineffective or inefficient.
In 2017, the figures varied from €172.99 to €293.37 per inhabitant. Spain had an average pharmaceutical expenditure of €218.56 per inhabitant (Source: MSCBS, MSSSI, and own elaboration).
Complementary border indicators:
PRESCRIPTIONS BY INHABITANT: (Number of total prescriptions)/(population resident in the community).
The main objective is to detect where they are prescribing more prescriptions than an adequate number.
In 2017, average prescriptions of up to 24.57 per inhabitant were reached, and the minimum was set at 15.07. The national average was 19.52 prescriptions per inhabitant (Source: MSCBS, MSSSI, and own elaboration).
HEALTH EXPENDITURE BY INHABITANT: (Total health expenditure)/(population resident in the community).
This indicates the total expenditure related to health per inhabitant, for later comparison (Source: MSCBS and own elaboration, 2017).
Once the simple index numbers are obtained, a synthetic indicator, IF, is calculated, which is the result of multiplying each index number (aging, chronicity, unemployment rate, and frequency) by its weighting.

4. Results and Discussion

As explained above, we have studied the collective behavior of the four elements considered: aging, chronicity, unemployment rate or level of income, and frequency of consultations in general medicine and pediatrics.
First of all, we started the analysis, including the unemployment rate (Table 1).
Next, to look for homogeneity in the comparison, four series of index numbers are constructed (one per indicator), taking as the base 100 the “national” value for each indicator considered (Table 2).
Once the simple index numbers are obtained, a synthetic indicator, IF, is calculated, which is the result of multiplying each index number (aging, chronicity, unemployment rate, and frequency) by its weighting. Likewise, the border index of pharmaceutical expenditure per inhabitant with a national base (IGFH) has been constructed (Table 3).
The two indices, IF and IGFH, can be represented in a scatter plot (Figure 3).
This comparative chart provides essential qualitative information on the relationship between the health and socio-economic environment or context, Pharmaceutical spending, and management efficiency.
The four quadrants allow positioning each REGION according to its context and pharmaceutical expenditure. This methodology could be applied to entire countries, provinces, or regions.
A region above the diagonal should focus its first efforts on getting as close as possible to the diagonal of points through pharmaceutical efficiency and the improvement of their systems. In a second step, the goal is positioning itself under the diagonal and as far as possible from it.
The Canary Islands and Andalusia are in quadrant IV (bottom right) with quite unfavorable conditions, but they are, on the other hand, able to be very efficient in their spending. In quadrant I (upper right) is the most sensitive group, formed by the Autonomous Communities that have an unfavorable context and that also have an above-average expense: Castile-La Mancha, Castile & León, Asturias, Galicia, and Extremadura, the latter standing out for its result, being very far to the right, very distant from the other autonomous communities. This reveals both their adverse contextual conditions and their problems in management. This group must pay special attention to the improvement of efficiency.
In quadrant II (upper left), the regions are located where the conditions are favorable but have an above-average expense: Aragón, Basque Country, Cantabria, Murcia, La Rioja, and the Valencian Community.
Finally, quadrant III (with a low cost and a favorable context) are the Balearic Islands, the Community of Madrid, Catalonia, and the Navarre.
Although the unemployment rate is an excellent indicator to analyze the level of wealth of a community, we have also used the average annual income per family (€). In the synthetic index, we have used the inverse of the level of income because we have assumed that the lower the income, the more public health systems are used, as they have fewer possibilities of using private alternatives. This new data set is presented, comparing it again with pharmaceutical expenditure per inhabitant (Table 4).
When changing context indicators, the synthetic indicator varies. Therefore, a new IF indicator is constructed (including family income and excluding the unemployment rate), and the new border analysis is carried out (Table 5).
And its corresponding graph (Figure 4).
If we consider the level of wealth instead of the unemployment rate, we find that some communities have changed their position in the graph.
First, some regions have changed quadrants. Catalonia has moved to the right side of the diagonal, placing itself in a more advantaged position than before, as well as closer to the average. The Canary Islands, on the other hand, has done the opposite, being placed in quadrant III (more favorable conditions) but above the diagonal. All the regions placed in quadrant II have moved to quadrant I. Castile & León and Galicia show improvements.
Second, we note that if we take these indices as a reference, the point cloud located around the national average is higher and closer, possibly explained by the fact that the differences in spending concerning the level of wealth are less significant.
As described above, for a more thorough and complete analysis, we have some more border indices that can be used to improve the efficiency and management of health systems.
The first case is taken as a reference, where the model has been made, including the unemployment rate. The indicator frontier number of prescriptions per inhabitant is analyzed first (Table 6).
And its corresponding graph (Figure 5).
The last constructed indicator takes the per capita health expenditure as a borderline model (Table 7).
And its corresponding graph (Figure 6).
Thus, if we study health expenditure per inhabitant, it happens that most regions are in the most efficient quadrant: Andalusia, Canary Islands, and Castile La Mancha have outstanding results. Besides, in this case, the point cloud is more dispersed than usual, a sign that there is a significant difference between the regions with the budget of health expenditure. It is worth highlighting the case of the Basque Country, which, in terms of health spending, allocates much more than other communities.

5. Conclusions

The epidemiological emergency of the coronavirus is putting the health systems at a global level at unbearable pressure, examining their effectiveness grow into a fundamental matter for health managers and policymakers. The context analysis proposed and developed in this paper has shown itself to be an appropriate way to measure the relative efficiency of public health management in comparison to other comparable health systems. Rational use of medicines requires that “patients receive medications appropriate to their clinical needs, in doses that meet their own individual requirements, for an adequate period of time, and at the lowest cost to them and their community” (WHO 2005). Especially, when there are not enough elements for a balanced comparison, either because the conditions of the socioeconomic and geographical environment are not equivalent, or because there are no other organizations that develop their activity in the same place and with the same users or clients. In these cases, the indicator that is taken as a border, for example: per capita pharmaceutical expenditure, must be abstracted from the context in order to make a fair comparison of the management and have a real measure of its relative efficiency. This methodology could be applied to entire countries, provinces, or regions. For example, some results indicate hospitals operating in a district with a high level of per capita public health expenditures experience gains in efficiency in comparison to hospitals in low spending districts (Hunt and Link 2020).
The methodology presented in this paper could help policymakers to make better decisions in order to improve the relative efficiency of the public pharmaceutical system, and it provides feedback on the decisions made in the past.
Specifically, in the case of pharmaceutical management studied in this article, the use of intensive variables, irrespective of the extension of the population, such as per capita expenditure, requires the evaluation of the context to become a well-calibrated indicator.
We have calculated two synthetic indicators. The first called IF, which is the result of multiplying each index number (aging, chronicity, unemployment rate, and frequency) by its weighting. The second is an index of pharmaceutical expenditure per inhabitant with a national base (IGFH).
A relevant conclusion is that although the unemployment rate is an excellent indicator to analyze the level of wealth of a community, we have also used the average annual income per family (€). In the synthetic index, we have used the inverse of the level of income because the lower the income, the more public health systems are used.
Health spending growth in OCDE countries is projected to grow in the coming years, reflecting population aging (La Maisonneuve et al. 2016), among other factors, such as the apparition of the COVID-19 global pandemic. For this reason, to find methodologies that help policymakers to assess the efficiency of pharmaceutical expenditures is a crucial point for worthwhile management of scarce health resources. This paper offers various indices to assess and help in management decisions about pharmaceutical expenditures by public health managers.
This paper suffers from some limitations. The proposed indicators are built upon the WAA method, which holds several drawbacks. Also, to select appropriate weighting and aggregation methods when constructing new indices is necessary (Gan et al. 2017). Future research could be to replicate these indices in other countries with different health backgrounds. Previous research has come together on the procedures to quantify socioeconomic inequalities, proposing that the selection of the inequality indicator has an intense effect on the volume and strength of the disparities analyzed (Harper et al. 2008; Polašek and Šogorić 2009).

Author Contributions

Conceptualization, J.M.V.-G. and M.M.-B.; Formal analysis, M.M.-B,, J.M.V.-G and F.J.R.-C.; Investigation, M.M.-B., J.M.V.-G. and F.J.R.-C.; Methodology, M.M.-B. and J.M.V.-G.; Visualization, M.M.-B., J.M.V.-G. and F.J.R.-C.; Writing—original draft, J.M.V.-G.; Writing—review and editing, M.M.-B. and F.J.R.-C. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A

We propose a multiple regression model with the border index of pharmaceutical expenditure per inhabitant as output and the simple indices of aging, chronicity, unemployment rate and frequentation as predictor variables. The results obtained are the presented in Table A1.
Table A1. R and ANOVA.
Table A1. R and ANOVA.
RR-SquaredAdj. R-SquaredStandard Error of Estimation
0.8550.7310.6416.9092112
ANOVA
ModelSum of SquaresdfMean SquareFSig.
Between Groups1553.2974388.3248.1350.002
Within Groups572.8461247.737
Total2126.14316
The model explains the 73.1% of the total variance, as you can see in Table A2.
Table A2. Partial Correlation Coefficient.
Table A2. Partial Correlation Coefficient.
Partial Correlation CoefficientWw
0.7311.8090.4041
0.2390.1321
0.6350.3510
0.2040.1128
Applying these weights for the construction of the synthetic context index, we obtain:
I F = i = 1 4 ω i   I i = 0.4041   I 1 + 0.1321   I 2 + 0.3510   I 3 + 0.1128   I 4
Table A3. Indices IF, Pharmaceutical Expenditure by Inhabitant, IGFH.
Table A3. Indices IF, Pharmaceutical Expenditure by Inhabitant, IGFH.
IFPharmaceutical Expenditure by InhabitantIGFHDistance to Bisector IGFH = IF
Spain100.00205.33100.00-
Andalusia111.48196.6895.7911.10
Aragón97.80220.09107.19−6.64
Asturias107.49238.17115.99−6.01
Balearic Islands80.91162.4979.141.26
Canary Islands103.65200.9297.854.10
Cantabria98.43209.93102.24−2.70
Castile-La Mancha109.51222.05108.140.97
Castile & León111.33225.06109.611.22
Catalonia91.63183.2989.271.67
Valencian Community97.86220.59107.43−6.77
Extremadura118.07278.07135.43−12.28
Galicia105.95247.89120.73−10.45
Madrid88.03170.8483.203.41
Murcia, Region of97.25215.78105.09−5.54
Navarre85.24193.0894.03−6.22
Basque Country92.49205.3199.99−5.30
La Rioja91.77204.1899.44−5.43
Figure A1. Comparison of the index IF vs. IGFH.
Figure A1. Comparison of the index IF vs. IGFH.
Admsci 10 00031 g0a1
Obviously, as you can see in Table A3 and Figure A1, whether the weights vary, the estimated efficiency change. If weights are assigned in this way to the different indicators that make up the synthetic index, although it also involves a certain degree of arbitrariness, it may be more realistic than arbitrary assignments.

References

  1. Abbott, Frederick M. 2005. The WTO Medicines Decision: World Pharmaceutical Trade and the Protection of Public Health. American Journal of International Law 99: 317–58. [Google Scholar] [CrossRef] [Green Version]
  2. Afonso, António, and Mina Kazemi. 2017. Assessing Public Spending Efficiency in 20 OECD Countries. In Inequality and Finance in Macrodynamics. Berlin: Springer, pp. 7–42. [Google Scholar]
  3. Anell, Anders. 2015. The Public–Private Pendulum—Patient Choice and Equity in Sweden. New England Journal of Medicine 372: 1–4. [Google Scholar] [CrossRef] [PubMed]
  4. Belloni, Annalisa, David Morgan, and Valérie Paris. 2016. Pharmaceutical Expenditure and Policies. Paris: OECD. [Google Scholar]
  5. CEPYME Aragón. 2011. Gestión de La Información y La Formación a Los Trabajadores de Las Pymes En Prevención de Riesgos Laborales. Available online: http://www.conectapyme.com/documentacion/GIF-PRL/recursos/contenidopdf/Nivel_5.pdf (accessed on 3 April 2019).
  6. Coelli, Timothy J., D S Prasada Rao, Christopher J. O’Donnell, and George E. Battese, eds. 2005. Productivity and Efficiency Measurement Concepts BT-An Introduction to Efficiency and Productivity Analysis. Boston: Springer US, pp. 41–83. [Google Scholar] [CrossRef]
  7. Domínguez Serrano, Mónica, Francisco Javier Blancas Peral, Flor María Guerrero Casas, and Mercedes González Lozano. 2011. Una Revisión Crítica Para La Construcción de Indicadores Sintéticos. Revista de Métodos Cuantitativos Para La Economía y La Empresa 11: 41–70. [Google Scholar]
  8. European Union. 2018. EN: Methodological Bases and Approach. Available online: https://europa.eu/capacity4dev/evaluation_guidelines/wiki/en-methodological-bases-and-approach-0 (accessed on 24 January 2019).
  9. Farrell, Michael James. 1957. The Measurement of Productive Efficiency. Journal of the Royal Statistical Society. Series A (General) 120: 253. [Google Scholar] [CrossRef]
  10. Freund, Rudolf J., William J. Wilson, and Donna Mohr. 2010. Statistical Methods, Students Solutions Manual (e-Only). Cambridge: Academic Press. [Google Scholar]
  11. Gan, Xiaoyu, Ignacio C. Fernandez, Jie Guo, Maxwell Wilson, Yuanyuan Zhao, Bingbing Zhou, and Jianguo Wu. 2017. When to Use What: Methods for Weighting and Aggregating Sustainability Indicators. Ecological Indicators 81: 491–502. [Google Scholar] [CrossRef]
  12. Häkkinen, Unto, and Isabelle Joumard. 2007. Cross-Country Analysis of Efficiency in OECD Health Care Sectors. Paris: OECD. [Google Scholar]
  13. Harper, S. John Lynch, Stephen C. Meersman, Nancy Breen, William W. Davis, and Marsha E. Reichman. 2008. An Overview of Methods for Monitoring Social Disparities in Cancer with an Example Using Trends in Lung Cancer Incidence by Area-Socioeconomic Position and Race-Ethnicity, 1992–2004. American Journal of Epidemiology 167: 889–99. [Google Scholar] [CrossRef] [Green Version]
  14. Heckley, Gawain, Ulf-G Gerdtham, and Gustav Kjellsson. 2016. A General Method for Decomposing the Causes of Socioeconomic Inequality in Health. Journal of Health Economics 48: 89–106. [Google Scholar] [CrossRef] [Green Version]
  15. Hunt, David J., and Charles R. Link. 2020. Better Outcomes at Lower Costs? The Effect of Public Health Expenditures on Hospital Efficiency. Applied Economics 52: 400–14. [Google Scholar] [CrossRef] [Green Version]
  16. Instituto Nacional de Estadística. 2015. Encuesta Europea de Salud en España 2014. Available online: https://www.ine.es/dyngs/INEbase/es/categoria.htm?c=Estadistica_P&cid=1254735976594 (accessed on 24 February 2019).
  17. La Maisonneuve, Christine De, Rodrigo Moreno-Serra, Fabrice Murtin, and Joaquim Oliveira Martins. 2016. The Drivers of Public Health Spending: Integrating Policies and Institutions. Paris: OECD. [Google Scholar]
  18. López Saludas, José Manuel López. 2013. Análisis de La Eficacia y Eficiencia Del Servicio Nacional de Salud. Boletín Económico De Ice 3044: 59–77. [Google Scholar]
  19. López, Jesús Clemente, Pedro García Castrillo, and María A González-Álvarez. 2019. Mortalidad y Estatus Socioeconómico En La España de Principios Del Siglo XXI. Investigaciones Regionales-Journal of Regional Research 5: 227–40. [Google Scholar]
  20. Martín, José J., and M. Puerto López del Amo. 2007. La Medida de La Eficiencia En Las Organizaciones Sanitarias. Presupuesto y Gasto Público 49: 139–61. [Google Scholar]
  21. Mazziotta, Matteo, and Adriano Pareto. 2013. Methods for Constructing Composite Indices: One for All or All for One. Rivista Italiana Di Economia Demografia e Statistica 67: 67–80. [Google Scholar]
  22. Méndez-Castrillón Susín, Javier. 2014. Análisis Del Sistema Sanitario Español: Estructura y Eficiencia. Madrid: Universidad Pontificia Comillas. [Google Scholar]
  23. Moscelli, Giuseppe, Luigi Siciliani, Nils Gutacker, and Richard Andrew Cookson. 2015. Socioeconomic Inequality of Access to Healthcare: Does Patients’ Choice Explain the Gradient? Evidence from the English NHS. Amsterdam: Elsevier, vol. 57, pp. 290–314. [Google Scholar]
  24. Ministerio de Sanidad Consumo y Bienestar Ssocial. 2018. Informe Anual del Sistema Nacional de Salud 2017. Available online: https://www.mscbs.gob.es/estadEstudios/estadisticas/sisInfSanSNS/tablasEstadisticas/InfSNS2017.htm (accessed on 15 January 2019).
  25. Mutis, Hernando. 2006. Una Aplicación Del Análisis de Frontera Estocástica: El Caso de Hospitales de Nivel II En Colombia. Lecturas Matemáticas, Volumen Especial 27: 259–70. [Google Scholar]
  26. Nardo, Michela, and Michela Saisana. 2008. OECD/JRC Handbook on Constructing Composite Indicators. Putting Theory into Practice. Brussels: European Commission-Joint Research Centre, Institute for the Protection and Security of the Citizen. [Google Scholar]
  27. Shireen Patel, Joe Kai, Christopher Atha, Anthony Avery, Boliang Guo, Marilyn James, Samuel Malins, Christopher Sampson, Michelle Stubley, and Richard Morriss. 2015. Clinical characteristics of persistent frequent attenders in primary care: Case–control study. Family Practice 32: 624–30. [Google Scholar]
  28. Parra, Rodríguez, and Francisco Javier. 2011. Análisis de Eficiencia y Productividad. Work document University of Cantabria. Cantabria: University of Cantabria. [Google Scholar]
  29. Polašek, Ozren, and Selma Šogorić. 2009. Socioeconomic Inequalities Estimation from Public Health Surveys–Data Considerations, Strengths and Limitations. Collegium Antropologicum 33: 171–76. [Google Scholar]
  30. Roemer-Mahler, Anne. 2013. Business Conflict and Global Politics: The Pharmaceutical Industry and the Global Protection of Intellectual Property Rights. Review of International Political Economy 20: 121–52. [Google Scholar] [CrossRef]
  31. Shabaninejad, Hosein, Mohammad Hossein Mirsalehian, and Gholamhossein Mehralian. 2014. Development of an Integrated Performance Measurement (PM) Model for Pharmaceutical Industry. Iranian Journal of Pharmaceutical Research: IJPR 13: 207. [Google Scholar]
  32. Simonet, Daniel. 2015. The New Public Management Theory in the British Health Care System: A Critical Review. Administration & Society 47: 802–26. [Google Scholar]
  33. Torchia, Mariateresa, Andrea Calabrò, and Michèle Morner. 2015. Public–Private Partnerships in the Health Care Sector: A Systematic Review of the Literature. Public Management Review 17: 236–61. [Google Scholar] [CrossRef]
  34. Varabyova, Yauheniya, and Julia-Maria Müller. 2016. The Efficiency of Health Care Production in OECD Countries: A Systematic Review and Meta-Analysis of Cross-Country Comparisons. Health Policy 120: 252–63. [Google Scholar] [CrossRef]
  35. Wagstaff, Adam. 1989. Estimating Efficiency in the Hospital Sector: A Comparison of Three Statistical Cost Frontier Models. Applied Economics 21: 659–72. [Google Scholar] [CrossRef]
  36. Wang, Ke, Jiayu Wang, Klaus Hubacek, Zhifu Mi, and Yi-Ming Wei. 2019. A Cost–Benefit Analysis of the Environmental Taxation Policy in China: A Frontier Analysis-based Environmentally Extended Input–Output Optimization Method. Journal of Industrial Ecology. [Google Scholar] [CrossRef]
  37. WHO. 2005. WHO|Rational Use of Medicines. Geneva: WHO. [Google Scholar]
Figure 1. Methodological scheme.
Figure 1. Methodological scheme.
Admsci 10 00031 g001
Figure 2. Comparison of the index context vs. frontier.
Figure 2. Comparison of the index context vs. frontier.
Admsci 10 00031 g002
Figure 3. Comparison of the index IF vs. IGFH.
Figure 3. Comparison of the index IF vs. IGFH.
Admsci 10 00031 g003
Figure 4. Comparison of the index IF vs. IGFH (recalculated).
Figure 4. Comparison of the index IF vs. IGFH (recalculated).
Admsci 10 00031 g004
Figure 5. Comparison of the index IF vs. IRPH.
Figure 5. Comparison of the index IF vs. IRPH.
Admsci 10 00031 g005
Figure 6. Comparison of the index IF vs. IGSH.
Figure 6. Comparison of the index IF vs. IGSH.
Admsci 10 00031 g006
Table 1. Analysis of aging, chronicity, unemployment rate, and level of income and frequency in general medicine and pediatrics.
Table 1. Analysis of aging, chronicity, unemployment rate, and level of income and frequency in general medicine and pediatrics.
AgingChronicityUnemployment RateFrequency in General Medicine (Primary Care)
Spain18.67%64.20%17.22%5.07
Andalusia16.61%64.16%25.51%5.65
Aragón20.70%66.95%11.65%5.34
Asturias24.52%72.89%13.71%4.90
Balearic Islands15.49%56.50%12.43%3.80
Canary Islands15.62%55.68%23.46%5.00
Cantabria20.76%56.46%13.56%5.16
Castile-La Mancha17.92%65.23%20.77%6.00
Castile& León23.86%66.08%14.08%6.58
Catalonia18.36%60.23%13.41%3.77
Valencian Community18.99%61.01%18.17%4.80
Extremadura19.63%53.64%26.22%6.82
Galicia24.43%79.30%15.67%5.72
Madrid17.12%65.33%13.34%4.76
Murcia15.50%64.78%18.03%5.48
Navarre18.89%67.15%10.24%4.75
Basque Country21.64%72.66%11.31%5.12
La Rioja20.12%61.19%12.00%5.71
Table 2. The weighting of the index numbers of aging, chronicity, unemployment rate, and level of income and frequency in general medicine and pediatrics.
Table 2. The weighting of the index numbers of aging, chronicity, unemployment rate, and level of income and frequency in general medicine and pediatrics.
The Weighting of the Index Numbers
25%25%25%25%
RAgingRCronicityRUnemployment RateRFrequency
Spain100.00100.00100.00100.00
Andalusia88.9599.94148.14111.44
Aragón110.87104.2867.65105.33
Asturias131.32113.5479.6296.65
Balearic Islands82.9888.0172.1874.95
Canary Islands83.6486.73136.2498.62
Cantabria111.2187.9478.75101.78
Castile-La Mancha95.98101.60120.62118.34
Castile & León127.77102.9381.77129.78
Catalonia98.3593.8277.8774.36
Valencian Community101.6895.03105.5294.67
Extremadura105.1583.55152.26134.52
Galicia130.85123.5291.00112.82
Madrid91.67101.7677.4793.89
Murcia, Region of83.03100.90104.70108.09
Navarre101.18104.6059.4793.69
Basque Country115.88113.1865.68100.99
La Rioja107.7895.3169.69112.62
Table 3. Indices IF, Pharmaceutical Expenditure by Inhabitant and IGFH.
Table 3. Indices IF, Pharmaceutical Expenditure by Inhabitant and IGFH.
IFPharmaceutical Expenditure by InhabitantIGFHDistance to Bisector IGFH = IF
Spain100.00218.57100.00-
Andalusia112.12209.5995.8911.48
Aragón97.03244.09111.68−10.36
Asturias105.28266.22121.80−11.68
Balearic Islands79.53173.0079.150.27
Canary Islands101.31217.7799.631.19
Cantabria94.92242.13110.78−11.21
Castile-La Mancha109.14243.32111.32−1.54
Castile & León110.56244.68111.95−0.98
Catalonia86.10190.7987.29−0.84
Valencian Community99.23249.91114.34−10.68
Extremadura118.87293.38134.23−10.86
Galicia114.55257.61117.87−2.35
Madrid91.19182.4383.475.46
Murcia, Region of99.18233.58106.87−5.44
Navarre89.73208.8495.55−4.12
Basque Country98.93223.34102.19−2.31
La Rioja96.35227.79104.22−5.56
Table 4. Analysis of Aging, Chronicity, Income Level (Inverse), Frequency in Medicine (Primary Care).
Table 4. Analysis of Aging, Chronicity, Income Level (Inverse), Frequency in Medicine (Primary Care).
AgingChronicityIncome Level (Inverse)Frequency in Medicine (Primary Care)
Spain18.67%64.20%0.000036287103565.07
Andalusia16.61%64.16%0.000042195873245.65
Aragón20.70%66.95%0.000034366623145.34
Asturias24.52%72.89%0.000036424564734.90
Balearic Islands15.49%56.50%0.000031091627023.80
Canary Islands15.62%55.68%0.000043878894255.00
Cantabria20.76%56.46%0.000037004144465.16
Castile-La Mancha17.92%65.23%0.000038295102066.00
Castile & León23.86%66.08%0.000043179757336.58
Catalonia18.36%60.23%0.000031835981033.77
Valencian Community18.99%61.01%0.000041607722394.80
Extremadura19.63%53.64%0.000049031625406.82
Galicia24.43%79.30%0.000037688915695.72
Madrid17.12%65.33%0.000030815691354.76
Murcia, Region of15.50%64.78%0.000042419614835.48
Navarre18.89%67.15%0.000029912356794.75
Basque Country21.64%72.66%0.000029237201425.12
La Rioja20.12%61.19%0.000034752389235.71
Table 5. Indices IF, Pharmaceutical Expenditure by Inhabitant, IGFH.
Table 5. Indices IF, Pharmaceutical Expenditure by Inhabitant, IGFH.
IFPharmaceutical Expenditure by InhabitantIGFHDistance to Bisector IGFH = IF
Spain100.00218.57100.00-
Andalusia112.12209.5995.8911.48
Aragón97.03244.09111.68−10.36
Asturias105.28266.22121.80−11.68
Balearic Islands79.53173.0079.150.27
Canary Islands101.31217.7799.631.19
Cantabria94.92242.13110.78−11.21
Castile-La Mancha109.14243.32111.32−1.54
Castile & León110.56244.68111.95−0.98
Catalonia86.10190.7987.29−0.84
Valencian Community99.23249.91114.34−10.68
Extremadura118.87293.38134.23−10.86
Galicia114.55257.61117.87−2.35
Madrid91.19182.4383.475.46
Murcia, Region of99.18233.58106.87−5.44
Navarre89.73208.8495.55−4.12
Basque Country98.93223.34102.19−2.31
La Rioja96.35227.79104.22−5.56
Table 6. Indices IF, Prescriptions by Inhabitant, IRPH.
Table 6. Indices IF, Prescriptions by Inhabitant, IRPH.
IFPrescriptions by InhabitantIRPHDistance to Bisector IRPH = IF
Spain100.0019.52100.00
Andalusia112.1220.32104.075.69
Aragón97.0319.96102.23−3.68
Asturias105.2821.04107.76−1.75
Balearic Islands79.5315.0777.211.64
Canary Islands101.3119.96102.22−0.64
Cantabria94.9219.0997.76−2.01
Castile-La Mancha109.1421.47109.97−0.59
Castile & León110.5621.41109.680.62
Catalonia86.1018.3093.75−5.41
Valencian Community99.2321.43109.75−7.44
Extremadura118.8724.57125.87−4.95
Galicia114.5523.03117.94−2.40
Madrid91.1916.1182.516.14
Murcia, Region of99.1820.47104.86−4.02
Navarre89.7318.2393.40−2.60
Basque Country98.9317.0287.188.31
La Rioja96.3519.4699.69−2.36
Table 7. Indices IF, Health Expenditure by Inhabitant, IGSH.
Table 7. Indices IF, Health Expenditure by Inhabitant, IGSH.
IFHealth Expenditure by InhabitantIGSHDistance to Bisector IGSH = IF
Spain100.001.332100.00
Andalusia110.831.11081.7120.59
Aragón101.211.54498.531.90
Asturias106.801.577115.49−6.14
Balearic Islands82.621.29196.82−10.04
Canary Islands102.511.308101.720.56
Cantabria102.431.446109.36−4.90
Castile-La Mancha112.141.30695.4711.79
Castile & León115.081.467107.505.36
Catalonia89.691.35991.49−1.27
Valencian Community95.951.32689.474.58
Extremadura119.051.549104.2510.47
Galicia101.111.42087.889.36
Madrid90.921.22491.91−0.70
Murcia, Region of101.461.49891.147.30
Navarre87.381.543118.24−21.82
Basque Country93.561.669128.57−24.76
La Rioja95.711.39890.443.73

Share and Cite

MDPI and ACS Style

Martínez-Blanes, M.; Rondán-Cataluña, F.J.; Velarde-Gestera, J.M. Indicators of Efficiency in the Pharmaceutical Management of a Public Health System. Adm. Sci. 2020, 10, 31. https://doi.org/10.3390/admsci10020031

AMA Style

Martínez-Blanes M, Rondán-Cataluña FJ, Velarde-Gestera JM. Indicators of Efficiency in the Pharmaceutical Management of a Public Health System. Administrative Sciences. 2020; 10(2):31. https://doi.org/10.3390/admsci10020031

Chicago/Turabian Style

Martínez-Blanes, Manuel, Francisco Javier Rondán-Cataluña, and José Manuel Velarde-Gestera. 2020. "Indicators of Efficiency in the Pharmaceutical Management of a Public Health System" Administrative Sciences 10, no. 2: 31. https://doi.org/10.3390/admsci10020031

APA Style

Martínez-Blanes, M., Rondán-Cataluña, F. J., & Velarde-Gestera, J. M. (2020). Indicators of Efficiency in the Pharmaceutical Management of a Public Health System. Administrative Sciences, 10(2), 31. https://doi.org/10.3390/admsci10020031

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop