The Use of the Data Envelopment Analysis–Malmquist Approach to Measure the Performance of Digital Transformation in EU Countries

Abstract

Currently, the process of the digital transformation of EU countries is very important and often discussed. It will not only bring new opportunities for companies and the broader population but will also enable the transition to a more ecological economy. An important goal is to speed up the digitalization processes taking place in companies. It is very important to use already established digitalization elements more efficiently. This also resulted in the motivation for the given research. The aim of this paper is to quantify the change in the efficiency of the digital transformation of EU countries. As part of this research, the Variable Returns to Scale Data Envelopment Analysis (VRS DEA) model and the Malmquist index (MI) based on the DEA approach were applied. The results of the model made it possible to assess how the changes in technical efficiency and technological changes contributed to the changes in efficiency. The long-term theoretical added value of this paper lies in its proposal for countries and their governments to monitor not only the number of introduced digital elements, but also the efficiency of their use relative to some aggregate output; for example, GDP (Gross Domestic Product) or unemployment rate. The added value of this research is that less developed countries use digitalization elements more effectively than developed countries.

1. Introduction

The current era is characterized by globalization, which penetrates into all areas of human life, and the economy is no exception in this regard. It is possible to point to various digital phenomena brought about by the digital economy. According to Stavytskyy et al. (2019), in general, the digital economy is not only the relationships mediated by the Internet, cellular communication and ICT (information and communication technologies). The digital economy also affects business interactions and speeds up the course of both a business itself and its various associated transactions. It also helps overcome barriers to entering the market and provides competitive advantages to companies and ultimately reduces costs. The digital economy is made up of economic activity that is the result of billions of daily online connections between people, businesses, devices, data and processes. The basis of the digital economy is hyperconnectivity, which means the growing interconnectedness of people, organizations and machines as a result of the Internet, mobile technologies and the Internet of things (Oloyede et al. 2023). The digital economy is constantly being formed and is determined by the structure and speed of the process of digitalization and digital transformation.

Nowadays, the issue of the digitalization and digital transformation of the European Union is highly topical. Differences in the digital transformation of EU countries are significant. “New” countries have difficulty catching up with the “old”. This is caused by both economic and social factors, such as levels of education, the age of the population and different marginal groups within countries, but also the geographical location of the countries and the settlement of individual parts of the territory. As part of the digitalization of the business environment, the problem of falling behind is caused by a lack of funds for the introduction of digitalization elements, especially for small and medium-sized companies, but also by distrust and fear associated with the security of company data. Many studies confirm the importance of digital transformation in relation to environmental and economic growth. However, from the point of view of individual EU countries, it is again necessary to point out the different levels of this perception and understanding. In the case of lagging countries, the awareness is not as strong as in the case of “old” countries that have been dealing with this issue for longer periods of time. There was a significant shift in the digitalization process of new countries as a result of the 2020 pandemic. Since this period, the level of understanding of the need to introduce digitization elements has been growing. However, it is necessary to popularize this need more within lagging countries.

In the context of monitoring digital progress, important reports on digitalization or digital transformation have been published. In 2021, the European Commission proposed a path to the Digital Decade, which is guided by the 2030 Digital Compass. This communication of the European Commission clearly sets priorities and thus ensures Europe’s successful digital transformation (Furlan 2022).

The targets and objectives of the 2030 Digital Agenda are structured according to four main areas: digital skills, digital infrastructure, digital transformation of businesses, and digitalization of public services (European Commission 2024a) (see

Table 1. Targets and objectives of the 2030 Digital Agenda.

Main Area	Targets and Objectives
Digital skills	ICT Specialists: 20 million + gender convergence
	Basic Digital Skills: min 80% of population
Digital infrastructure	Connectivity: gigabit for everyone
	Cutting edge Semiconductors: double EU share in global production
	Data—Edge and Cloud: 10,000 climate-neutral highly secure edge nodes
Digital transformation of businesses	Tech up-take: 75% of EU companies using Cloud, AI, or Big Data
	Innovators: grow scale-ups and finance to double EU Unicorns
	Late adopters: ensure more than 90% of SMEs reach at least a basic level of digital intensity
Digitalization of public services	Key Public Services: 100% online
	e-Health: 100% of citizens have access to medical records online
	Digital Identity: 100% of citizens have access to digital ID

Source: European Commission (2024b).

). The core indicators of the DESI, which is one of the key tools for monitoring and measuring digital transformation in EU member countries, were aligned with these goals in 2021 (Bittner et al. 2022).

Another major policy initiative is the Recovery and Resilience Instrument. According to the European Commission, together with the Digital Compass, these are two main political initiatives that will have an impact on digital transformation in the EU. Through the Recovery and Resilience facility, the Commission obtains funds by borrowing from the capital markets. These funds can be used by Member States to enhance their green and digital transition in line with the EU’s priorities (European Commission 2024d).

The main challenge comes within the business environments of Industry 4.0 and Industry 5.0, which represent an important motivator in the field of digitalization. In connection with this industrial revolution, it is possible to point out challenges in relation to educational institutions, which are forced to transform their programs in order to prepare quality employees for the intelligent environment. High-quality staff and high-quality software solutions are prerequisites for increasing the efficiency of the digitalization process. It is important that every person develops their digital skills and participates in the online world.

The driving force behind this research was the fact that there are not many studies (Inel 2019; Bánhidi et al. 2021; Rejman Petrovic et al. 2022; Kao et al. 2022; Georgescu et al. 2022; Lungu et al. 2022; Krstić et al. 2023) in the given area that focus on the application of the DEA method, as well as on the application of the Malmquist index (Bozkurt et al. 2022), which is based on the use of this non-parametric method. When choosing indicators to evaluate digital transformation, this research was inspired by Digital Compass indicators as well as DESI indicators. To measure the performance of the economy, the GDP per capita and unemployment rate were chosen. The relationship between digitalization indicators and GDP or labor market indicators was studied by several authors. Parra et al. (2020) studied the link between the variables of the DESI and GDP per capita. The results of their study revealed that “the implication between technology indicators and GDP per capita not only takes place but is also relevant specifically to aspects linked to the use of internet services by citizens and in the integration of digital technology by technology businesses” (Parra et al. 2020, p. 168). The influence of the DESI on labor market indicators was studied by Basol and Yalcin (2021). The results of their study confirmed that “an increase in the DESI has increased employment rate and personal earnings, which are perceived as positive labor market indicators and have decreased long-term unemployment rate and labor market insecurity, which are negative labor market indicators” (Basol and Yalcin 2021, p. 503).

Based on the above-mentioned studies, the aim of this study was to diagnose the efficiency of the digital transformation of EU countries using the MI in relation to their economic growth. In relation to this aim, we set the following research question: Is the efficiency of digitalization of developed countries higher than the efficiency of less developed countries?

The remainder of this paper is structured as follows: The “Literature Review” Section 2 defines digitalization, digital economy and digital efficiency. The “Research Methodology” Section 3 specifies the source of the data, describes the theoretical basis of the DEA and Malmquist Productivity Index based on the DEA approach and lists the applications of DEA models in the subject area or research. The “Results” Section 4 offers the results of the efficiency of EU countries in the use of digitalization elements using the VRS DEA model. It also contains the results of the change in efficiency over time calculated with the use of the Malmquist index. The “Discussion” Section 5 compares the results of the DESI with the results achieved by the VRS DEA model and the Malmquist index. The “Conclusion” Section 6 offers conclusions, limitations and future research.

2. Literature Review

The process of digitalization is essential to ensure the economic growth of EU countries. This path is particularly important for businesses in individual countries, as the economic growth of these businesses and their sustainability is a prerequisite for the growth of the country’s economy. According to Sabbagh et al. (2012), digitalization offers economic growth; countries at the most advanced stage of digitalization gain 20% more economic benefits than countries at the initial stage. Digitalization has a demonstrable impact on reducing unemployment, improving quality of life and improving citizens’ access to public services. Finally, yet importantly, digitalization allows governments to work with greater transparency and efficiency. In relation to digitalization and its impact on economic growth, it is appropriate to mention the term digital economy. The term “digital economy” refers to the convergence of computing and communications technologies on the Internet and the resulting flow of information and technology that is stimulating all electronic commerce and vast organizational change. No current public policy issue is more timely or has the potential to affect more people (Lane 1999). The “digital” economy is at the forefront of efficiency gains: the more customers a company has, the more “effective” it will be at attracting new customers and others to the production process, and will be able to offer better services at the same price. The benefits of a “digital” economy are many: most importantly, it will prevent shadow economies and help eliminate corruption (Jurayevich and Bulturbayevic 2020). According to Mesenbourg (2001), digital economy has three primary components: e-business infrastructure, electronic business (e-business) and electronic commerce (e-commerce). Many definitions of digital economy have arisen over time. However, many of them are simple and straightforward variants of the definition of the European Commission, which defines a digital economy as “an economy based on digital technologies” (Bukht and Heeks 2019, p. 4). Barefoot et al. (2018) understand the digital economy in terms of the introduction of the Internet and other information technologies. The digital economy is a kind of economy characterized by the active implementation and actual application of digital technologies for the collection, storage, processing, transformation and transmission of data in absolutely all areas of human activity (Borremans et al. 2018). The terminology “digital economy” mostly denotes the recent and significantly unrealized changes in various segments of the economy by the computer-assisted digitalization of data (Williams 2021). The term digital economy has been introduced “to support the improvement of the national economy and the rapid development of knowledge and technology at this time” (Limna et al. 2022, p. 3).

Measuring and comparing the level and efficiency of the digital transformations taking place in countries is a crucial issue. Therefore, the next part of this text lists studies addressing this problem.

In his research, Yalcin (2021) examined the effectiveness of the digitization of EU countries. The aim of this research was to identify countries that transformed digitalization into their growth and employment rates in the period 2015–2019. In his study, the following dimensions of the DESI of the EU-28 Member States were used as inputs: Connectivity, Human Capital, Use of Internet, Integration of Digital Technology and Digital Public Services; the GDP Growth Rate and Employment Rate were used as outputs (Yalcin 2021). The results of the study show that “in countries such as Bulgaria, Cyprus, Hungary and Romania, the effective use of DESI indicators, which are considered a measure of digitalization, has a one-on-one effect on the GDP and ER values. DESI indicators are not used effectively in the GDP and ER values of countries like Denmark, Finland, and Spain. In general, in terms of digitalization, it has been revealed that developing countries use digitalisation more effectively in terms of economic growth and job creation” (Yalcin 2021, p. 323). The correlations between digitization and economic growth in EU countries were analyzed by Georgescu et al. (2023). They applied Principal Component Analysis and neural networks to analyze the digitalization indicators obtained from Eurostat and the World Bank for the period 2019–2021. With the use of this combined approach, they intended to enhance the classification accuracy of EU countries classified according to the DESI. The results revealed an improvement in classification accuracy. The importance of building an effective digital economy infrastructure in order to improve international competitiveness was pointed out by Balcerzak and Pietrzak (2017). Their research compared the development level of digital economies in V4 countries with the application of the TOPSIS method. The results of the research confirmed relatively quick progress in building digital economies at the regional level in all V4 countries. However, the research revealed significant disparities between the analyzed regions, especially Polish regions. Malkowska et al. (2021) investigated the impact of digital transformation on EU countries. The study compared the technological development of EU countries based on three dimensions: the digitalization of society, the ability of the economy to face the challenges of technological development and the exploitation of ICT in companies. The applied methods included cluster analysis and the TOPSIS method. With the use of these methods, EU countries were grouped according to similar levels of technological development into countries with high, medium and low performance. This helped to identify the digitalization gaps between countries.

Recently, the Data Envelopment Analysis method has come to the fore in many areas of research. Emrouznejad and Yang (2018) estimate that there are up to 10,300 papers focusing on the application of DEA models in various fields. It is possible to give examples of the application of DEA models, e.g., Paradi et al. (2004) applied the DEA method to manufacturing enterprises. Feruś (2010) applied the DEA model to construction companies. Kohl et al. (2019) applied the DEA method to hospitals. The DEA method was applied to measure the efficiency of banks by Jablonsky (2012), Sorayaei et al. (2012) and Akther et al. (2013). Paradi et al. (2014) focused on efficiency analysis in services. Sreedevi (2016) analyzed the efficiency of public service companies using the DEA model. Lim and Zhu (2019) focused on the application of DEA in the field of insurance companies. Gúčik and Uličná (2014) applied the DEA model and the Malmquist test to analyze the efficiency of hotel facilities. Kocher et al. (2006) applied the DEA method in economic research across countries. The input data consisted of the expenditure on research in the given area, the number of universities with departments of economics and, as an uncontrollable input, the number of inhabitants. The output was the number of outputs in the top 10 journals for the years 1980–1998. The research was carried out in 21 OECD countries. The USA was the most efficient country when applying a constant returns to scale (CRS) model, and the USA, New Zealand and Ireland were the most efficient when applying the BCC model (Kocher et al. 2006). Jurickova et al. (2019) used Data Envelopment Analysis to measure the technical efficiency of national innovation systems in a sample of European countries. In this study, the numbers of researchers and the expenditures on research and development were used as inputs and published scientific journal articles and applied patents were used as outputs of the DEA model. The results showed that the most efficient countries in 2016 were Cyprus, Luxembourg, Malta and Romania. Germany, which is the EU best performer in terms of patent ranking, was classified as inefficient with an efficiency score of 0.5. The authors concluded that there are differences between the innovation performance examined by various indices available in public databases and the technical efficiency measured by DEA. The impact of digitalization on economic growth explained by total factor productivity (TFP) was investigated by Bozkurt et al. (2022). They calculated TFP based on the Malmquist index using labor, capital and gross domestic capital data for 30 countries including EU Member States. To determine the impact of digitalization on TFP, they used a tobit panel estimate. The results revealed a significant and positive impact of digitalization on TFP. Therefore, the authors concluded that digitalization contributes to productivity.

Over time, DEA also began to be applied when dealing with the efficiency of digital transformation. Inel (2019) examined the efficiency of the digital transformation of EU countries using the DEA method. The data necessary for the research was taken from the Digital Transformation Scoreboard 2018. ”Digital Infrastructure, Investment and Access to Finance, Supply and Demand of Digital Skills, E-leadership and Entrepreneurial Culture are considered as inputs, while ICT start-ups and Digital Transformation are considered as the output of DEA model. The results indicate that while some countries like Denmark, Italy and United Kingdom are considered relatively efficient, the Netherlands and Germany are not very efficient” (Inel 2019, p. 549). Krejnus et al. (2023) measured the efficiency of e-Government digitalization within the EU by applying DEA. For the inputs and outputs for the DEA model, they used attributes from the following digitalization and e-Government indices: GII (Global Innovation Index), Internet usage, DSGI (Digital Skills Gap Index), interaction with public administration online and e-Government benchmarks. They concluded that when comparing European regions, the countries of Northern Europe were the most efficient at e-Government digitalization. Bánhidi et al. (2021) compared the development of digitalization in Russia with EU Member States. They used data from the International Digital Economy and Society Index (I-DESI 2018) database. Their main research questions were concerned with the robustness of the EU data supply and the stability of its ranking. To answer them, they used Data Envelopment Analysis and a one-dimensional version of multidimensional scaling. Based on a comparison of the results, they suggested that the applied methods provided a similar solution, but the ranking of some countries (including Russia) showed wider variation. Rejman Petrovic et al. (2022) investigated the intensity and results of the digitization process in 2016–2019 in Serbia. They applied the DEA method in their research. The authors pointed to the fact that in Serbia, the digitization process was effective; the only drawback was the insufficient use of software packages. The authors also pointed out the lack of studies carried out in the given subject area.

Based on the above-mentioned studies, we can state that most studies have focused on the use of the DESI and the analysis of the development of digitalization from the point of view of the number of introduced digitalization elements. Other studies (Bánhidi et al. 2020; Borowiecki et al. 2021) have evaluated the static position of countries and their ranking in the field of digital transformation over one year. There are only a few articles published in the given area that have applied the DEA methodology. Moreover, according to the information currently available to the authors, the Malmquist index, which allows dynamic development to be analyzed, has only been used in the related field in the study of Bozkurt et al. (2022). This study aims to fill this research gap by comparing the development of the digital transformation of countries over time with the use of a DEA–Malmquist technique. The results of this technique make it possible to assess how the changes in technical efficiency and technology contributed to the changes in efficiency. In such a way, they differ from the results of other quantitative methodologies used to measure the efficiency of digital transformation in previous studies.

3. Research Methodology

In this empirical study, the efficiency of the use of digitalization elements in EU countries in the years 2019, 2021 and 2023 was assessed using the VRS DEA model. The change in the efficiency of the EU countries’ digital transformation was calculated using the Malmquist index, which is based on the application of DEA over a period of time. The Principal Component Analysis (PCA) method was applied to graphically illustrate the position of the EU countries in the digitalization process in 2023.

DEA is a non-parametric method that can be used to measure the efficiency of a group of peer units, which are called Decision-Making Units (DMUs). This method was applied in various countries and regions to measure the efficiency and performance of various units, such as banks, universities, manufacturing enterprises, hospitals and insurance companies (Cooper et al. 2011). The DEA method is based on the efforts of Farrell (1957) to develop a better method of productivity evaluation (Cooper et al. 2011). Farrell proposed a new approach to efficiency measurement based on the use of a convex curve and functions for the measurement of the distance between the monitored enterprise and the projected point on the efficiency frontier. In this way, he proposed a new way of measuring business efficiency based on the calculation of two components of overall business efficiency: technical efficiency and allocative efficiency (Färe et al. (1984)).

The original ideas of Farrell (1957) were later reformulated by Charnes et al. (1978), who created the CCR model. The authors described DEA as a mathematical programming model that provides a new way of obtaining empirical estimates of relationships between observed data. Subsequently, this model was transformed by Banker et al. (1984) into the BCC model. The aim of the mentioned methods is to eliminate or exclude subjectivity by measuring the effectiveness of the outputs in relation to the inputs.

For research in the given area, the VRS DEA model was applied to calculate the efficiency of the digital transformation of EU countries. We solved the dual output-oriented VRS DEA model which can be defined as follows (1) (Zhu 2014):

$$\begin{array}{c}max \varnothing +\epsilon \left({\displaystyle \sum _{i=1}^m}{s}_i^{-}+{\displaystyle \sum _{r=1}^s}{s}_r^{+}\right)\hfill \\ \mathrm{subject} \mathrm{to}\hfill \\ {\displaystyle \sum _{j=1}^n}{\lambda }_j{x}_{ij}+s_i^{-}=x_{io}\hfill & i=\mathrm{1,2},\dots ,m;\hfill \\ {\displaystyle \sum _{j=1}^n}{\lambda }_j{y}_{rj}-s_r^{+}=\varnothing y_{ro}\hfill & r=\mathrm{1,2},\dots ,s;\hfill \\ {\displaystyle \sum _{j=1}^n}{\lambda }_j=1\hfill \\ {\lambda }_j\ge 0\hfill & j=\mathrm{1,2},\dots ,n,\hfill \end{array}$$

(1)

where $\varnothing$ is the value of the objective function, $\epsilon$ is the non-Archimedean infinitesimal value, $x_{ij}, i=\mathrm{1,2},\dots ,m, j=\mathrm{1,2},\dots ,n$ is the value of the $i$ input for ${DMU}_i$, $y_{rj}, r=\mathrm{1,2},\dots ,s, j=\mathrm{1,2},\dots ,n$ is the value of the $k$ output for ${DMU}_j$. ${DMU}_0$ is one of the $n$ DMUs under evaluation, $x_{i0}$ and $y_{ro}$ are the i input and r output for ${DMU}_0$, respectively, $m$ and $s$ are the numbers of inputs and outputs, respectively, ${\lambda }_j$ is the convex coefficient, and $s_i^{-}$ and $s_r^{+}$ are the input and output slacks, respectively.

To assess countries’ efficiency over time, the Malmquist index was applied. This index measures the change in the total factor productivity (TFP) between two periods based on the principle of calculating the ratio of the distances of each data point relative to a common technology. The index is based on the use of distance functions, which allow us to describe production technology with multiple inputs and outputs without the need to specify the objectives of the company’s behavior. It is possible to define both input and output distance functions (Fandel 2002).

Several methods can be used to estimate distance functions, which are the starting points for calculating TFP. Linear programming is most often used, especially the Data Envelopment Analysis (DEA) method proposed by Färe, Grosskopf, Norris and Zhang in 1994 (Fandel 2002). The use of DEA models to calculate TFP is related to the assumption that homogeneous data are available over several periods. Calculating the change in efficiency requires solving four linear programming problems, assuming the use of technology with constant returns to scale (Fandel 2002).

Suppose that each ${DMU}_j\left(j=\mathrm{1,2},\dots ,n\right)$ uses a vector of inputs $x_j^t=\left(x_{1j}^t,\dots ,x_{mj}^t\right)$ to produce the vector of the outputs $y_j^t=\left(y_{1j}^t,\dots ,y_{mj}^t\right)$ at each period of time $t,t=1,\dots ,T$. The efficiency of ${DMU}_0$ can change or the frontier can shift, or both changes may occur at the same time. The Malmquist Productivity Index is then defined as follows (2) (Zhu 2014):

$${MI}_o={\left[{\displaystyle \frac{{\theta }_o^t\left(x_o^t, y_o^t\right)}{{ \theta }_o^t\left(x_o^{t+1}, y_o^{t+1}\right)}}{\displaystyle \frac{{\theta }_o^{t+1}\left(x_o^t, y_o^t\right)}{ {\theta }_o^{t+1}\left(x_o^{t+1}, y_o^{t+1}\right)}}\right]}^{{\displaystyle \frac{1}{2}}}$$

(2)

where ${MI}_0$ measures the change in productivity between the periods $t$ and $t+1. {\theta }_o^t\left(x_o^t, y_o^t\right)$ is calculated by comparing $x_o^t$ to the EPF (Empirical Production Frontier) at the time $t$ with the use of the input-oriented CRS DEA model, while $x_o^t$ = ($x_{1o}^t,\dots \dots \dots \dots x_{mo}^t$) and $y_o^t$ = ($y_{1o}^t,\dots \dots \dots \dots y_{so}^t$) are the input and output vectors of ${DMU}_0$, among others. Similarly, ${\theta }_o^{t+1}\left(x_o^{t+1}, y_o^{t+1}\right)$ is calculated by comparing $x_o^{t+1}$ to the EPF at the time t + 1, ${\theta }_o^{t+1}\left(x_o^t, y_o^t\right)$ by comparing $x_o^t$ to the EPF at time $t+1$; and ${\theta }_o^t\left(x_o^{t+1}, y_o^{t+1}\right)$ is calculated by comparing $x_o^{t+1}$ to the EPF at the time $t$, applying the input-oriented CRS DEA model.

This model is also available in its modified form (3) (Zhu 2014):

$${MI}_o={\displaystyle \frac{{\theta }_o^t\left(x_o^t, y_o^t\right)}{{ \theta }_o^{t+1}\left(x_o^{t+1}, y_o^{t+1}\right)}}\times {\left[{\displaystyle \frac{{ \theta }_o^{t+1}\left(x_o^{t+1}, y_o^{t+1}\right)}{{ \theta }_o^t\left(x_o^{t+1}, y_o^{t+1}\right)}}{\displaystyle \frac{{\theta }_o^{t+1}\left(x_o^t, y_o^t\right)}{{\theta }_o^t\left(x_o^t, y_o^t\right)}}\right]}^{{\displaystyle \frac{1}{2}}}$$

(3)

According to Fandel (2002), the term (4) represents a change in technical efficiency; it is the efficiency known according to Farrell (1957). The change in efficiency is equivalent to the ratio of Farrell’s efficiency at time $t$ and at time $t+1$ (technical efficiency change between the periods $t$ and $t+1$). Usually, this term expresses the improvement, deterioration or stability of technical efficiency.

$$TECH={\displaystyle \frac{{\theta }_o^t\left(x_o^t, y_o^t\right)}{{ \theta }_o^{t+1}\left(x_o^{t+1}, y_o^{t+1}\right)}}$$

(4)

The term (5) represents the frontier shift (FS) in the EPF between the periods $t$ and $t+1$ (Zhu 2014).

$$FS={\left[{\displaystyle \frac{{ \theta }_o^{t+1}\left(x_o^{t+1}, y_o^{t+1}\right)}{{ \theta }_o^t\left(x_o^{t+1}, y_o^{t+1}\right)}}{\displaystyle \frac{{\theta }_o^{t+1}\left(x_o^t, y_o^t\right)}{{\theta }_o^t\left(x_o^t, y_o^t\right)}}\right]}^{{\displaystyle \frac{1}{2}}}$$

(5)

The values of the indicators can be interpreted as follows:

▪

$TECH>1$: the efficiency of the DMS improved, it approached the production possibility frontier and the decisions taken were correct;

▪

$TECH=1$: the efficiency of the DMS did not change;

▪

$TECH<1$: the efficiency of the DMS declined; the decisions taken were incorrect.

TECH informs us about a change in technical efficiency but does not inform us about a shift in the production possibility frontier as a result of technological change. This is reported by the frontier shift as follows:

▪

$FS>1$: the efficiency increased and the frontier shifted outwards;

▪

$FS=1$: the efficiency frontier did not shift;

▪

$FS<1$: the efficiency declined and the frontier shifted inwards. This index informs us about a change within the entire EU.

Indicators representing 4 areas of the Digital Compass were selected as input data to fulfill the aim of this paper and to assess the effectiveness of the digital transformation and its impact on GDP and the unemployment rate. The following inputs were used: individuals’ level of digital skills—individuals with basic or above basic overall digital skills (all five component indicators at basic or above basic level) (DG); employed ICT specialists—percentage of total employment (ICTS); households with access to the Internet at home—percentage of households (HAIHs); Internet used—percentage of individuals; cloud computing services—percentage of enterprises; enterprises using DSL or other fixed broadband connections—percentage of enterprises, small enterprises (10–49 employees and self-employed persons), without the financial sector; e-Government users—percentage of internet users (last 12 months). GDP per capita and the unemployment rate were used as outputs. The data of EU-27 Member States as well as the EU average were collected for the years 2019, 2021 and 2023. Descriptive statistics for the selected indicators for the year 2023 are presented in

Table 2. Descriptive statistics of EU-27 Member States’ indicators for the year 2023.

Variable	Descriptive Statistics
	Mean	Median	Minimum	Maximum	Std. Dev.
Digital skills	57.89	58.95	27.73	82.70	13.05
Employed ICT specialists	4.07	3.90	2.20	6.80	1.25
Households with access to the Internet at home	93.20	93.22	86.90	99.06	3.14
Internet used	92.55	92.75	83.97	99.40	4.64
Cloud computing services	46.25	45.60	13.60	87.70	18.16
Enterprises using DSL or other fixed broadband connections	93.85	96.20	76.10	100.00	5.72
e-Government users	79.71	82.97	23.55	98.86	18.31
GDP per capita	29,080.00	25,160.00	7850.00	83,320.00	18,322.23
Total unemployment rate	5.74	5.55	2.54	12.04	2.29

Source: processed by authors in Statistica 14.1.0.8 software.

. The data were obtained from Eurostat (2024).

The average proportion of the EU population with basic or above basic overall digital skills in 2023 is 57.89%. In total, 4.07% of the total EU population are represented by ICT specialists. Around 7% of households still do not have an Internet connection. Cloud computing services are used on average by only 46.25% of companies in the EU. Especially in this area, it is necessary to develop activities towards increasing the use of these services. It is also necessary to improve e-Government activities, which are at the level of 79.71% in EU countries. Another negative phenomenon is the higher average unemployment rate of EU countries.

4. Results

The starting point of this research was a comparison of the results of the implementation of seven selected indicators of the digitalization process for the year 2023. As can be seen from Figure 1, compared to the selected countries, Slovakia achieved the worst results in almost all analyzed areas, except for digital skills, employed ICT specialists and cloud computing services. It is clear from the graph that Slovakia’s results reflect the EU average. The Netherlands and Finland achieved the best results in terms of digital skills, Sweden achieved the best results in terms of ICT specialists, Luxembourg achieved the best results in terms of HAIHs and Luxembourg achieved the best results in terms of Internet use. Cloud computing services were best used by Finland and Sweden. In terms of enterprises using DSL and e-Government users, Denmark dominated.

The position of the EU countries from the point of view of the selected digital indices, GDP and unemployment rate, is shown in Figure 2. This portfolio is one of the possible graphical outputs of the PCA method, which allows a large number of indicators to be replaced with several factors. Factor 1 and factor 2 in the portfolio define a window into the K-dimensional variable space. By projecting all the observations onto the low-dimensional sub-space and plotting the results, they allow us to visualize the structure of the analyzed data set. Factor 1 is highly correlated with the indicators digital skills, employed ICT specialists, households with access to the internet at home, Internet use, cloud computing services, e-Government users and GDP per capita. Factor 2 is correlated with the indicators of enterprises using DSL or other fixed broadband connections and total unemployment rate. The portfolio consists of four quadrants. In quadrant A, the countries that report problems in the field of fulfilling criteria laid down by individual digital indices, as well as in “GDP” and “Unemployment rate”, are shown. The best countries in the given field are located in quadrant B. This group includes countries like Ireland, Austria, Netherlands, Czechia and Malta. The Nordic countries Sweden, Finland and Denmark are very close to the border limits. Some of these countries show minor shortcomings in the fields “Individuals’ level of digital skills”, “Enterprises use DSL and other fixed broadband connection” and “Unemployment rate”. Quadrant D includes the countries that are in the worst position from the point of view of not only all digital transformation indices, but also “GDP” and “Unemployment rate”. These include Greece, Italy, Portugal and Cyprus.

The results of the DEA VRS model for the years 2019, 2021 and 2023 are presented in Figure 3. From this figure, it is clear that Bulgaria, Czechia, Germany, Ireland, Greece, Italy, Latvia, Lithuania, Luxembourg, Hungary, Poland, Portugal and Romania achieved an efficiency score of 1 in each year. This shows the effective impact of the use of digitalization elements on the results of individual countries expressed by an increase in GDP and a decrease in the unemployment rate; i.e., these countries lie on the efficiency frontier. In other countries, efficiency results grow over time, and in 2023, Croatia, Malta, and Slovakia also reached the efficiency frontier. The results show that some countries that led the field in introducing digitalization elements do not lie on the efficiency frontier. This can be justified by the lower rate of introduction of individual digitalization elements, since these elements were already introduced in the given countries in a larger volume and thus their impact on GDP growth and the unemployment rate is not so significant. This mainly concerns Finland, Sweden, the Netherlands, Austria, Denmark, Belgium, etc.

To assess the efficiency over time, the Malmquist index for the periods 2019–2021 and 2021–2023 was applied. The results of the MI for the period 2019–2021 are presented in

Table 3. Results of the MI for the years 2019/2021.

No.	DMUs in Period 1	Input-Oriented CRS Malmquist Index	Efficiency Change	Frontier Shift
1	EU-27 countries	1.04773	0.98455	1.06417
2	Belgium	1.03035	0.94648	1.08861
3	Bulgaria	1.33122	1.00000	1.33122
4	Czechia	1.02675	0.93622	1.09670
5	Denmark	1.00629	0.94844	1.06099
6	Germany	1.00061	0.94839	1.05506
7	Estonia	1.03101	0.93373	1.10419
8	Ireland	0.99220	1.00000	0.99220
9	Greece	1.05749	1.00000	1.05749
10	Spain	1.06549	1.00442	1.06080
11	France	1.05456	0.97275	1.08410
12	Croatia	1.06777	0.97589	1.09415
13	Italy	1.07582	0.98751	1.08943
14	Cyprus	1.02988	0.98471	1.04587
15	Latvia	1.08555	1.00000	1.08555
16	Lithuania	1.07319	1.00497	1.06787
17	Luxembourg	1.01838	1.00000	1.01838
18	Hungary	1.06959	1.00000	1.06959
19	Malta	1.05963	0.96026	1.10348
20	Netherlands	1.00471	0.94866	1.05909
21	Austria	1.05217	1.00560	1.04631
22	Poland	1.17378	0.96995	1.21015
23	Portugal	1.07369	1.00112	1.07248
24	Romania	1.17226	1.00000	1.17226
25	Slovenia	1.04205	0.96194	1.08328
26	Slovakia	1.06848	0.99875	1.06981
27	Finland	1.01587	0.94815	1.07142
28	Sweden	0.99103	0.93907	1.05533

Source: processed by authors in DEAFrontier software.

. Technical efficiency at the required level 1 has been achieved in almost every EU country. In our opinion, this was caused by a shift in the efficiency frontier, as a result of the technological changes that occurred in individual countries over the years. Since this is a comparison of the years 2019 and 2021, it can be assumed that these significant technological changes also occurred due to the impact of the COVID-19 pandemic. The change in the efficiency of the use of already established digitalization elements was equal to 1 only in some countries (see Table 3). When comparing the years 2019 and 2021, these countries used previously established digitalization elements very effectively (Bulgaria, Ireland, Spain, Latvia, Lithuania, Luxembourg, Hungary, Austria, Portugal, Romania).

If we compare the results of the MI for the years 2021/2023, we can see that the technical efficiency in some countries fell below 1 (see

Table 4. Results of the MI for the years 2021/2023.

No.	DMUs in Period 2	Input-Oriented CRS Malmquist Index	Efficiency Change	Frontier Shift
1	EU-27 countries	1.00488	0.99267	1.01229
2	Belgium	1.01877	1.01833	1.00043
3	Bulgaria	1.05546	1.00000	1.05546
4	Czechia	1.04176	1.01427	1.02711
5	Denmark	0.99026	0.99636	0.99388
6	Germany	1.00010	1.00000	1.00010
7	Estonia	1.01604	0.98863	1.02773
8	Ireland	1.08906	1.00000	1.08906
9	Greece	0.96663	1.00000	0.96663
10	Spain	0.96256	0.94443	1.01920
11	France	0.98929	0.97734	1.01222
12	Croatia	1.01408	0.97246	1.04280
13	Italy	1.06680	1.00382	1.06274
14	Cyprus	0.97628	0.96336	1.01341
15	Latvia	0.95732	1.00000	0.95732
16	Lithuania	1.00459	0.99264	1.01204
17	Luxembourg	1.18784	1.00000	1.18784
18	Hungary	1.06688	1.02158	1.04434
19	Malta	1.01732	0.99684	1.02054
20	Netherlands	0.99680	0.97805	1.01916
21	Austria	0.97966	0.95902	1.02153
22	Poland	1.02660	1.00000	1.02660
23	Portugal	1.02133	0.99449	1.02699
24	Romania	1.19876	1.00000	1.19876
25	Slovenia	0.99727	0.96942	1.02873
26	Slovakia	0.99769	0.98073	1.01730
27	Finland	1.00428	0.99719	1.00711
28	Sweden	1.00628	1.00831	0.99799

Source: processed by authors in DEAFrontier software.

). These were mainly Denmark, Greece, Spain, France, Cyprus, Latvia, Austria, Slovakia and Slovenia. This was caused both by a decrease in the efficiency of the use of already introduced digitalization elements, as well as the introduction of new digitalization elements. For example, in the case of Sweden, we can see that the MI is above 1. The change in efficiency, i.e., the use of previously introduced digitalization elements, is also above 1. However, the shift in the efficiency frontier is below 1, which means that the shift in the digitalization of this country is no longer necessary.

Figure 4 shows that the most significant changes in technical efficiency occurred when comparing the years 2019/2023, especially in Bulgaria, Italy, Hungary, Poland and Romania. These results again show that the countries that were particularly lagging behind in the digitalization process have been involved in it in recent years.

The change in the efficiency of the use of already introduced digitalization elements was the most significant when comparing the years 2021/2023 (see Figure 5). This change was more pronounced in Belgium, Czechia, Denmark, Germany, Estonia, the Netherlands, Finland and Sweden. These countries are trying to make the most of the previously established digitalization elements and are trying to use them effectively.

The most significant changes in frontier shifts occurred when comparing the years 2019 and 2023. The largest technological changes were implemented in Bulgaria, Italy, Hungary, Poland and Romania (see Figure 6).

The best results in introducing technological changes were achieved by Bulgaria and Romania. These results indicate that these countries are catching up with the leaders in the implementation of digitalization elements. In the field of efficiency change, several countries maintained a value of 1 (Bulgaria, Ireland, Greece, Latvia, Hungary, Romania). It is clear that the leaders in the field of introducing digitalization elements have completed this process with a high % of fulfillment of the criteria and, as a result, are not accelerating compared to other countries.

5. Discussion

Based on a review of the literature, it can be concluded that there are several approaches dedicated to the assessment of the digital transformation of EU countries. The analyzed studies use different methods and different indicators to assess this process, including methods such as TOPSIS (Balcerzak and Pietrzak 2017; Bánhidi et al. 2021), cluster analysis (Zaharia and Bălăcescu 2020; Malkowska et al. 2021; Bánhidi et al. 2020) and the fixed-effects methodology (Parra et al. 2020). The DEA method was applied to determine the efficiency of digital transformation by Inel (2019), Bánhidi et al. (2021) and Georgescu et al. (2022). Indicators based on the DESI (Zaharia and Bălăcescu 2020; Parra et al. 2020; Bánhidi et al. 2020; Georgescu et al. 2022), but also others that are not based on DESI (Balcerzak and Pietrzak 2017; Zaharia and Bălăcescu 2020; Malkowska et al. 2021) were used as input parameters. In most studies, it was mainly the annual development of digitalization indicators that was measured (Bánhidi et al. 2020; Borowiecki et al. 2021). It was also pointed out that digitalization has an impact on increases in the GDP of countries (Parra et al. 2020; Corejova and Chinoracky 2021; Olczyk and Kuc-Czarnecka 2022).

In this paper, seven digitalization inputs selected based on the Digital Compass and DESI were selected. The efficient use of these inputs was evaluated in relation to GDP and the unemployment rate, which were selected as outputs. The position of countries according to the selected inputs and outputs changed over time. Our comparison between the years 2019 and 2023 is shown in Figure 7. We can see that Ireland, Austria, the Netherlands and Czechia were placed in quadrant B in both periods. These countries were among the best ones in the analyzed years. The positions of Belgium, Denmark and Estonia, which were placed in quadrant C, slightly worsened. Even though the use of most digitalization elements in these countries increased, their unemployment rates either did not change or even increased. Germany moved from quadrant B to quadrant A, which was caused by the deterioration of the indicator “Households with access to the internet at home”. Malta improved its position and moved from quadrant A to quadrant B, placing it among the best countries. This is due to a significant improvement in all digitalization indicators as well as in GDP and unemployment rate. Luxembourg moved from quadrant C to quadrant A, so in its case there was an improvement in the unemployment rate and a deterioration in the indicator “Individuals’ level of digital skills”. There were slight improvements in Spain and Lithuania, especially with regard to the indicator “Enterprises use DSL or other fixed broadband connection”. Some countries, for example, Greece, did not show a significant shift.

In this paper, the digitalization process of individual EU countries was assessed from the point of view of changes in the efficiency of digital transformation using the MI and the VRS DEA model. Such an approach, namely the application of the MI, brings a new perspective to enhancing the digitalization process. Therefore, it is appropriate to supplement the assessment of countries and their rankings based on the quantity of introduced digitalization elements with a new perspective, which is the assessment of this process as an efficiency index. With the use of the MI, it is also possible to find out if shifts in efficiency during some periods was achieved mainly by the more effective use of previously introduced digitalization elements or by introducing new digital technologies. In the following part of the discussion, we provide some significant findings in the given area.

Based on the application of the MI, we found that the best results were achieved when comparing the years 2019/2023 in Bulgaria, Italy, Hungary, Poland and Romania. The positive results of the MI in these countries were caused mainly by introducing new digitalization elements (see Figure 6). However, it should be noted that the most important leaders from the point of view of the number of introduced digitalization elements measured by the DESI—Finland, Denmark, the Netherlands and Sweden (European Commission 2024c)—did not achieve the best results according to the MI. These countries achieved MI values close to 1 in both analyzed periods, so we can say that they maintained a high level of previously introduced digitalization elements. These results cannot be compared with previous studies because we did not find a study in which the MI was applied to analyze the efficiency of digital transformation. In the research of Bozkurt et al. (2022) mentioned in the Introduction, the MI was applied in a slightly different way. It was used to measure total factor productivity (TFP) using labor, capital and gross domestic capital data, and then the impact of digitalization on TFP was analyzed using a different method.

The VRS DEA model showed similar results. The countries that were in each year included Bulgaria, Czechia, Germany, Ireland, Greece, Italy, Latvia, Lithuania, Luxembourg, Hungary, Poland, Portugal and Romania. Croatia, Malta and Slovakia also reached the efficiency frontier in 2023. Countries that are leaders in the implementation of digitalization elements according to the DESI 2022 like Finland, Denmark, the Netherlands and Sweden (European Commission 2024c) were evaluated as being ineffective by the VRS DEA model. This can be justified by the lower rate of introduction of individual digitalization elements, since these elements are already introduced in the given countries in a larger volume and thus their impact on GDP growth and the unemployment rate is not so significant. These results more or less confirm the findings of Inel (2019) and Yalcin (2021). Slightly different results can be found in the study by Georgescu et al. (2022). These authors found that the best results of efficiency were achieved by Denmark, Estonia, Italy, Finland, Latvia, Lithuania and Malta. The Netherlands and Sweden did not achieve efficiency. We can conclude that when analyzing countries from the point of view of their digital transformation, it is important to evaluate not only the quantity of introduced digitalization elements, but also the effectiveness of their use in relation to GDP growth and employment growth in the country.

The position of the EU countries from the point of view of the MI, TECH and FS when comparing the years 2021 and 2023 is shown in the following 3D scatterplot (Figure 8). The best results were achieved by Ireland, Italy, Hungary and Bulgaria. These countries are closely followed by Czechia and Poland. Slovakia achieved the best results in the frontier shift—a shift in the field of technological innovation. The worst MI results were achieved by Latvia, Greece and Estonia.

6. Conclusions

The DEA model and the MI, which were applied in this study using the Digital Compass and DESI parameters for the years 2019, 2021 and 2023, provided an evaluation of the efficiency of 27 EU countries. We can conclude that the most significant changes in technical efficiency measured by MI occurred when comparing the years 2019/2023 in Bulgaria, Italy, Hungary, Poland and Romania. The positive results of the MI in these countries were caused mainly by introducing new digitalization elements. These countries were also effective in each year based on the DEA results. The change in the efficiency of the use of already introduced digitalization elements was the most significant when comparing the years 2021–2023 in Belgium, Czechia, Denmark, Germany, Estonia, the Netherlands, Finland and Sweden. Some countries like Czechia or Germany were also effective in each year based on the DEA results. However, Denmark, the Netherlands, Finland and Sweden, which are the most important leaders from the point of view of the number of introduced digitalization elements measured by the DESI 2022, were not effective based on the results of the VRS DEA model. The results from the perspective of these changes are different. This is due to the fact that some countries are at the forefront due to their better use of already established elements of digitalization and some due to the introduction of new technologies, while in the case of these countries it is a more intensive process.

The digitalization solutions using the DEA VRS and MI showed different results compared to the evaluation of the digitalization process, for example, using DESI. This finding is also confirmed by the study of Georgescu et al. (2022). This is due to the fact that the MI calculates the change in efficiency over two periods. On the other hand, most digitalization indices are based on the digital transformation data achieved in one year. Another reason for the above may be the fact that due to the limitations of the DEA model, only a limited number of digitalization elements could be used. It is also necessary to point out the limitations regarding the quantity and quality of data. Eurostat does not capture data for all years and data for some countries are also missing. That is why, for example, the year 2022 is missing from this analysis. The data must be carefully searched for in various reports and studies, and therefore only those indicators for which data can be found can be selected.

The results show that the MI has a higher informative value when it measures the effectiveness of the transformation process over time instead of for a given year. Changes over time were confirmed, especially in the countries located in the middle of the ranking. There are studies (Inel 2019; Yalcin 2021) which, using the DEA models, confirmed that some lower-ranking EU countries already established digital processes with more efficiency than the ranking leaders (also in terms of the impact on economic growth and employment). This statement was also confirmed in this study by the VRS DEA model and MI and answered the set research question. However, it is obvious that lower-ranking EU countries are still lagging behind in the introduction of new digitalization elements. The main pillar of the countries’ digital policies is the possibility of equal access to technologies and innovative possibilities. This is the only way to ensure equality within EU countries. Currently, the world is approaching the end of the introduction phase of the “ICT age”, and the introduction phase of a new paradigm, including Industry 4.0 technologies, is beginning. It is necessary to prevent these technologies from further exacerbating inequalities within countries. Much will depend on whether countries catch up, advance or fall behind, which will depend on national policies. EU policy makers should encourage greater knowledge exchange across the EU, with structured programs that allow for the sharing of best practices and digital solutions between EU countries.

The results of this study significantly enrich the knowledge in the methodology of investigating the efficiency of digitalization of EU-27 countries, and from this point of view they represent a significant theoretical contribution to the given issue. In addition, the findings of this study provide theoretical as well as practical applications, especially in terms of building an innovative knowledge economy. They point to the diversity of the EU-27 countries in digitalization and suggest ways to eliminate it, either by better use of the existing digitalization elements or by intensifying the introduction of new ones. Diagnosing the level of digitalization and established levels of the use of innovative technologies among EU-27 enterprises can ensure a comprehensive approach to the digital transformation process, taking into account the differences between these countries. Within this process it is necessary to focus on more consistent data collection and to introduce new ways and methods of evaluating the achieved results, especially by applying mathematical programming as well as artificial intelligence; namely, the implementation of artificial intelligence tools lags behind in both developed as well as less developed countries. Therefore, governments, institutions and organizations that deal with the given issue should tackle it more consistently and take into account the results of scientific research. A comprehensive approach to solving this issue could include the MI score—an innovative index of the digital transformation of EU countries over time, which will take into account the level as well as the efficiency of the digitalization of EU countries. This could also be a challenge within the governments of individual countries to focus not only on the introduction of digitalization elements, but also their effective use. The benefit of this research for practice is the finding that digitalization is beneficial for increasing the performance of countries and businesses, but it is necessary to take into account the increase in expenses at the beginning of the introduction of individual digitalization elements.

Future research will be aimed at confirming the finding that less developed EU countries use elements of the digitalization process more effectively than developed countries. The goal is also to assess the effectiveness of the digitalization process of businesses in EU countries, the impact of this process on the performance of businesses’ and averting the risk of their bankruptcy.