Citizens’ Perception of Digital Public Services: A Case Study among Romanian Citizens

Abstract

Digital transformation has become a topic of interest since public institutions digitalize their processes to improve efficiency, while citizens wish to solve their issues through online platforms. Even if this topic is a priority, the Digital Economy and Society Index reveals that digital public services from UE are under 30% development and implementation. This paper presents the current research made on digital public services and the perspective of Romanian citizens on the adoption of these platforms. The authors conduct bibliometric research to highlight the current progress and concerns. Furthermore, a quantitative analysis is made to determine the perception of Romanian citizens on digital public services and the factors that influence their decision to use them instead of traditional methods. According to the bibliometric analysis, 23 countries showed a raised interest for digital public services after 2020, which was a crucial moment that pushed digitalization among public institutions. Although Romanian citizens are aware of the benefits brought by digital public services, they encounter issues in terms of lack of information and technical support, knowledge, data security, and the complexity of these platforms. Therefore, the government should implement an integrated approach that considers cultural values as well as reliable and accessible digital solutions.

1. Introduction

Digitalization of organizations is a developing process, enhancing transparency and accessibility of services. Depending on the resources available and the specific context, organizations aim to integrate digital technologies across all activity areas to the greatest extent possible, aiming to improve functionality.

While digitalization offers numerous advantages in information processing and management, shaping the relationship between organizations and individuals is slow. According to the DESI index (European Commission 2024), which encompasses four main areas—human capital, connectivity, integration of digital technology, and digital public services—there has been a marginal increase over the last decade. Romania occupies the last place in Europe, being the country where digital public services are least prevalent.

These digital public services are intended to create a link between state organizations and citizens by translating forms, requests, payments, and physical activities into the digital realm via web platforms. A primary focus of this article is enhancing the scientific angle in digital public services, along with findings from research conducted in the last five years.

This facilitates an evaluation of the current state in the e-governance and open innovation domains. To understand the factors behind the digitalization ranking of countries—particularly Romania’s position—the perspectives of Romanian citizens are indispensable. They represent one half of the state–citizen relationship, contributing to the DESI index.

Platforms have been adopted by state organizations as detailed at (ADR 2023), with ongoing projects aimed at digitizing public institutions in Romania. Nonetheless, the existence of these platforms has not led to a substantial increase in the DESI index, which remains low. Therefore, further measures and procedures are needed to engage and enhance digital public services. Another aspect this article addresses is identifying the factors and perspectives of Romanian citizens regarding the use of digital public services, to pinpoint current challenges and develop a framework for future research.

Digital public services are smart public services, also known as e-government. They employ technology to provide local, regional, and national residents with services. These services make it easier, faster, and less expensive for individuals, companies, and organizations to communicate with governmental administrations.

There are still citizens without digital skills at a basic level, and this affects their participation in public life in an online environment (De Filippi et al. 2020). To cope with this, strategies to develop e-government tools are promoted by the European Union (EU). An important part is represented by creating applications and digital platforms user-centered to support citizens and engage them (Faber et al. 2020). To have public participation and an embedded governance is widely spread, and one of the key elements is related to co-production behavior (Nasifuhan et al. 2024). As technology is advancing, civic engagement becomes digital, along with their interactions (Bassetti et al. 2023).

More and more governments have used technologies for communication to implement digital platforms and allow citizens an environment for e-participation. Here, an important contribution is given by Information and Communication Technology (ICT) regulation and usage (Adam and Dzang Alhassan 2022). The relationship between the citizens and the state has become mediated by technology, and governments aim to strengthen citizens’ engagement in using public digital tools and platforms to pursue the direction of smart cities and e-government (Gagliardia et al. 2021).

Thus, in a world governed by technology and digital tools, public organizations must adapt to meet citizens and EU requirements. By identifying the current issues and results in the field of digital public services, one can have a complete overview and identify possible solutions to increase and improve the reachability and usage of digital platforms provided by the government. Considering this context, a bibliometric research and quantitative research was conducted, to determine the factors that influence citizens to embrace digital public services.

2. Materials and Methods

The aim of this research is to determine the factors that engage Romanian citizens into using digital public services, analyzing their perception and attitude as well as evaluating trends and research directions in this field. By identifying these factors, researchers and stakeholders may provide solutions to improve the landscape of e-government by offering improved access and a better user experience.

To achieve the objective of this paper (Figure 1), there were established two secondary objectives. The first one is to determine the current state of research in digital public services, through bibliometric analysis. Thus, it is searched the evolution of scientific papers in this field as well as the distribution of the articles. This question can clarify the aim of the performed analysis by identifying trends, geographical concerns, and the impact of the research made so far on this field. The second objective is to determine the perspective of Romanian citizens toward the usage of digital public services using quantitative research. Here, the question is which are the main factors that influences the attitude of Romanian citizens on adopting digital public services. By establishing these factors, analysis can be performed on them, and correlations may be drawn. A deeper understanding on Romanian citizens perception is gained that is relevant to propose methods that support the improvement of digital public services to meet their expectations.

To carry out this research, the first step consisted in the analysis of scientific articles of the bibliometric research type, to identify the structure. The documentation regarding the VOS Viewer program, version 1.6.20, and the functionalities offered by it follow.

The authors carried out the bibliometric research in June 2024, using the scientific database Web of Science, being a complex one, with free access offered by higher education institutions. This database was chosen because it includes the highest-quality publications, both at the level of conferences and magazines from all over the world. The Web of Science database has numerous filters, which allow researchers to refine their search and customize it in the desired direction.

In the research of this work, keywords were chosen to include scientific articles regarding the use of digital platforms offered by the state to citizens worldwide: citizen engagement, digital public platforms, government online platforms, digital public services, e-government. Because we wanted to document the citizens’ perspective on public digital platforms, the following criteria were used:

I.

Inclusion criteria:

I.1.

All articles that include the previously mentioned keywords as a topic as follows:

I.1.1.

Citizen engagement and digital public platforms OR

I.1.2.

Citizen engagement and government online platforms OR

I.1.3.

Citizen engagement and digital public services OR

I.1.4.

Citizen engagement and e-government

I.2.

Scientific works such as Articles and Proceedings

I.3.

The research should be published between 2020 and 2024, because the COVID-19 pandemic has accelerated the digitization process of public institutions and the creation of digital platforms that allow the interaction between the citizen and the state.

I.4.

Articles published in English.

II.

Exclusion criteria

II.1.

Other databases

II.2.

Articles published before 2020 because technology is advancing rapidly, and this research aims to analyze the latest trends and research carried out.

In the first stage, the keywords determined by the authors provided 687 papers published and indexed in ISI Web of Science. Only scientific paper from the last 5 years were selected, which allowed an up-to-date analysis. This resulted in a sample of 290 papers, which follows the changes generated by the COVID-19 pandemic, as well as the desire for digitization necessary in the current context, dominated by technology and automation, artificial intelligence, and big data. Selecting the document type Article and Proceeding papers resulted in 271 papers, and being written in English led to a final number of 261. A model for this research is presented in Figure 2. All these documents were downloaded using Export as a tab delimited file, including for each content record author, title, source, having a txt format. This type of format allows data processing with the help of the VOS Viewer tool, to create graphs and analyze the results obtained.

The second part of the study is represented by the quantitative research. To analyze the factors that determine the acceptance and use of digital public services, the TAM—Technology Acceptance Model (Davis 1985, 1993; Davis et al. 1989) was used as a starting point, which emphasizes two important factors that influence the intention to use a technology: perceived usefulness and perceived ease of use (Figure 3). Various studies in the literature also mention that these two variables are influenced by various external variables (Venkatesh and Davis 1996; Davis et al. 1989; Venkatesh and Bala 2008).

Thus, considering the results of a focus group research (Barbu et al. 2023), as well as the information identified in the literature (

Table 1. The factors considered in the study.

Factors	Sources
Perceived Usefulness (PU)	(Vărzaru 2022; Fauzi et al. 2022; Kang et al. 2021; Kamal et al. 2020; Xie et al. 2017)
Perceived Ease of Use (PEU)
Behavioral Intention to Use (BIU)
Attitude Toward Using (ATU)
Digital Inclusion and Diversity Approach (DIDA)	(Al-Muwil et al. 2019; López et al. 2018; Almuwil 2014)
Citizens’ Technological Competencies (CTTC)	(Barbu et al. 2023; Reissig et al. 2022; Teeroovengadum et al. 2017)
Social influence (SI)	(AlHadid et al. 2022; Balaskas et al. 2022; Alarabiat et al. 2021; Venkatesh et al. 2012; Chan et al. 2010)
Data Security (DS)	(Barbu et al. 2023; Omar et al. 2023; Balaskas et al. 2022; Kang et al. 2021)
Data Privacy (DP)	(Barbu et al. 2023; Omar et al. 2023; Kamal et al. 2020)
Level of citizen engagement and participation (LCEP)	(Barbu et al. 2023; Alarabiat et al. 2021; Choi and Song 2020)
Resistance to use (RU)	(Méndez-Rivera et al. 2023; Kamal et al. 2020; Alomari 2014)
Citizen satisfaction and experience in interacting with digital public services (CSEI)	(Barbu et al. 2023; Vărzaru 2022; Chan et al. 2021; Al-Muwil et al. 2019)
Perceived Risk (PR)	(Song and Jo 2023; AlHadid et al. 2022; Kamal et al. 2020; Xie et al. 2017)
Facilitating Conditions (FC)	(AlHadid et al. 2022; Kamal et al. 2020; Saxena 2017; Setiawati and Pratiwi 2015; Riffai et al. 2012; Chan et al. 2010)
Technical Accessibility and Digital Infmutructure (TADI)	(Barbu et al. 2023; Song and Jo 2023; Fauzi et al. 2022; Balaskas et al. 2022; Saxena 2017)
Technology Anxiety (TA)	(Kamal et al. 2020; Guner and Acarturk 2020; Kamal et al. 2020)
Cultural values and attitudes towards technology (CVTAT)	(Khan 2022; Kang et al. 2021; Zhao et al. 2021)
Transparency and communication from public authorities (TCPA)	(Barbu et al. 2023; Choi and Song 2020; Srivastava 2011)

), several external variables were taken into account to be included in the TAM model to be tested (Figure 4).

Thus, based on these consulted sources, the authors consider that these factors are important and could represent external variables that can be considered when applying the extended TAM model (Figure 4). For example, the variable Social Influence (SI) was considered because it refers to the impact that the opinion of other people (family, colleagues, community) has on the decision to use technology, and a positive social influence could improve BIU and ATU (AlHadid et al. 2022; Balaskas et al. 2022; Alarabiat et al. 2021; Venkatesh et al. 2012; Chan et al. 2010). Users’ perception of data security (Data Security—DS) can significantly influence PU and PEU, since if citizens consider a technology safe, then they could be more encouraged to use it (Barbu et al. 2023; Omar et al. 2023; Balaskas et al. 2022; Kang et al. 2021). In addition, if the technology respects user privacy (Data Privacy—DP), then it can increase PU and ATU (Barbu et al. 2023; Omar et al. 2023; Kamal et al. 2020). Moreover, perceived risks must also be considered, reflecting users’ fears related to the use of technology (Perceived Risk—PR), since a lower perceived risk can increase PU and PEU (Song and Jo 2023; AlHadid et al. 2022; Kamal et al. 2020; Xie et al. 2017).

The technological aspect must also be considered when analyzing the level of adoption of digital public services. First, Technical Accessibility and Digital Infrastructure (TADI) is important, as a solid technical infrastructure can improve technology accessibility (Barbu et al. 2023; Song and Jo 2023; Fauzi et al. 2022; Balaskas et al. 2022; Saxena 2017) and, implicitly, PEU. But also, the rest of the resources and the support available for the use of technology is important (Facilitating Conditions—FC), as favorable conditions can improve PEU and BIU (AlHadid et al. 2022; Kamal et al. 2020; Saxena 2017; Setiawati and Pratiwi 2015; Riffai et al. 2012; Chan et al. 2010).

Technology Anxiety (TA) can reduce PEU and BIU (Kamal et al. 2020; Guner and Acarturk 2020; Kamal et al. 2020). Reducing TA level through education can support acceptance. Thus, increasing the level of users’ technical skills (Citizens’ Technological Competencies—CTTC) can increase PEU and reduce technological anxiety (Barbu et al. 2023; Reissig et al. 2022; Teeroovengadum et al. 2017), thus increasing technology acceptance. However, we must also consider Resistance to Use (RU), which reflects negative attitudes toward technology adoption (Méndez-Rivera et al. 2023; Kamal et al. 2020; Alomari 2014). A high RU resistance can decrease PU, PEU, and, by implication, BIU.

However, since citizens live in a complex society where all kinds of external and internal factors intervene for individuals, cultural values (Cultural Values and Attitudes Towards Technology—CVTAT) can influence the way users perceive usefulness and ease of use, affecting thus the acceptance of technology (Khan 2022; Kang et al. 2021; Zhao et al. 2021). The authorities have an important role here, as clear and transparent communication from them (Transparency and Communication from Public Authorities—TCPA) can increase users’ trust in technology (Barbu et al. 2023; Choi and Song 2020; Srivastava 2011), improving PU and ATU. In addition, the authorities should promote the accessibility of technologies (Al-Muwil et al. 2019; López et al. 2018; Almuwil 2014) for all social groups (Digital Inclusion and Diversity Approach—DIDA), an approach that can improve PU and PEU by ensuring that technologies are adapted to the diverse needs of users. Thus, a positive user experience (Citizen Satisfaction and Experience in Interacting with Digital Public Services—CSEI) can increase ATU and therefore BIU, contributing to a greater acceptance of digital services (Barbu et al. 2023; Vărzaru 2022; Chan et al. 2021; Al-Muwil et al. 2019). At the same time, a greater commitment from citizens (Level of Citizen Engagement and Participation—LCEP) can lead to a more positive perception of the usefulness and ease of use of the technology (Barbu et al. 2023; Alarabiat et al. 2021; Choi and Song 2020).

To test the model, quantitative research was carried out, the main objective of which is to determine the factors that influence the intention to use and accept digital public services. In order to collect the data necessary for this analysis, a questionnaire was created with 28 items that were grouped into three sections (Supplementary Material): one section deals with the experience and general perception of the respondents associated with the digitization of public services, one section deals with various factors that can influence the adoption of digital public services, and the last section collects demographic data and control variables. The variables from the first section analyzed the existence of interaction between respondents and digital public services, the future intention to use digital public services, the types of platforms or mobile applications associated with national public services used by respondents, the types of public services accessed using digital methods in the last two years, and the advantages and disadvantages of the digitalization process of public services. In the second section, 18 factors were analyzed: Perceived Usefulness (PU), Perceived Ease of Use (PEU), Behavioral Intention to Use (BIU), Attitude Toward Using (ATU), Digital Inclusion and Diversity Approach (DIDA), Citizens’ Technological Competencies (CTTC), Social influence (SI), Data Security (DS), Data Privacy (DP), Level of citizen engagement and participation (LCEP), Resistance to use (RU), Citizen satisfaction and experience in interacting with digital public services (CSEI), Perceived Risk (PR), Facilitating Conditions (FC), Technical Accessibility and Digital Infrastructure (TADI), Technology Anxiety (TA), Cultural values and attitudes toward technology (CVTAT), Transparency and communication from public authorities (TCPA). For these types of variables, respondents’ perceptions were measured, using a 5-point Likert scale, from Total Disagreement (1) to Total Agreement (5). For the demographic and control variables, the following variables were considered: gender, age, the last level of education completed, the environment of origin, and the type of electronic devices used.

To determine the main factors that influence the level of adoption and use of digital public services, a pilot study was carried out targeting the citizens of sector 6 of the city of Bucharest. Thus, the developed questionnaire was published on a discussion group of the social media page related to the Bucharest Sector 6 City Hall, a page that had 38,000 followers. The questionnaire could be accessed by all citizens who had access to that social media group, regardless of age, gender, or socioeconomic status. To encourage participation in completing the questionnaire, the form sent had a short description, to be easy to read, emphasizing its importance to the community. Also, the message in the description of the post contained the statement “Your participation is essential for the improvement of public services in the community,” the authors considering that this statement could make citizens contribute with their answers to solving these problems. The post was visible between 5 and 30 July 2024, being viewed by 1278 followers, of which only 84 of them completed the proposed questionnaire. Unfortunately, the response rate was very low; from the total group it represented 0.22%, but from the total number of people who viewed the message, the response rate was 6.57%. This low ratio can suggest a low level of community involvement and interest in research or local issues. If the message was posted by a known local representative or directly by the city hall, maybe it would have shown a greater potential interest in response and involvement.

The items proposed to describe the factors that were used in this quantitative research were tested for consistency using Cronbach alpha coefficients. According to Hair et al. (2007), the coefficient should have a minimum value of 0.7, while other researchers (Fornell and Larcker 1981; Garson 2010) suggested a minimum of 0.6. However, values that range between 0.7 and 0.8 are adequate, those between 0.8 and 0.9 are very good, and all above are excellent, according to Kline (2005).

The internal consistency of the set of criteria is presented in Table 1 where it is observed that all the variables, except the TADI and FC factors, have a Cronbach alpha coefficient above 0.7, which reflects a good internal consistency of the items. However, Van Griethuijsen et al. (2014) mention that a factor above 0.6 still reflects an acceptable consistency of these considered items, which shows that the FC factor also has good internal consistency of the items.

Table 2. Cronbach’s alpha coefficients of the tested items.

Analyzed Factors	Cronbach’s Alpha	No. Items	Average	Variation	Standard Deviation	The Formula for Obtaining Items
PU	0.873	4	17.68	9.708	3.116	PU = (PU1 + PU3 + PU2 + PU4)/4.
PEU	0.886	4	16.64	14.386	3.793	PEU = (PEU1 + PEU2 + PEU3 + PEU4)/4.
ATU	0.94	4	17.57	15.069	3.882	ATU = (ATU1 + ATU2 + ATU3 + ATU4)/4.
BIU	0.887	4	17.25	10.935	3.307	BIU = (BIU1 + BIU2 + BIU3 + BIU4)/4.
TCPA	0.907	4	11.82	18.152	4.261	TCPA = (TCPA1 + TCPA2 + TCPA3 + TCPA4)/4.
LCEP	0.715	3	12.07	6.661	2.581	LCEP = (LCEP1 + LCEP2 + LCEP3)/3.
CVTAT	0.833	3	14.04	3.517	1.875	CVTAT = (CVTAT1 + CVTAT2 + CVTAT3)/3.
RU	0.966	4	7.43	28.254	5.315	RU = (RU1 + RU2 + RU3 + RU4)/4.
DIDA	0.91	3	14.29	3.841	1.96	DIDA = (DIDA1 + DIDA2 + DIDA3)/3.
CSEI	0.9	3	11.89	6.396	2.529	CSEI = (CSEI1 + CSEI2 + CSEI3)/3.
DS	0.957	4	13.71	14.508	3.809	DS = (DS1 + DS2 + DS3 + DS4)/4.
DP	0.849	2	6.89	4.099	2.025	DP = (DP1 + DP3)/2.
PR	0.749	4	16.39	8.692	2.948	PR = (PR1 + PR2 + PR3 + PR4)/4.
FC	0.648	3	10.71	4.508	2.123	FC = (FC2 + FC3 + FC4)/3.
CTT	0.959	4	19.14	5.905	2.43	CTT = (CTT1 + CTT2 + CTT3 + CTT4)/4.
TA	0.864	4	8.29	13.397	3.66	TA = (TA1 + TA2 + TA3 + TA4)/4.
SI	0.869	3	10.18	12.226	3.497	SI = (SI1 + SI2 + SI3)/3.

also shows the ways to obtain the tested factors.

3. Results

3.1. The Perspective of Scientific Researchers toward Digital Public Services

The field of digital public services is analyzed by carrying out a bibliometric analysis, so that all relevant articles from the last 5 years are considered. This interval was chosen, as technology advances rapidly and undergoes constant changes, which is why the new practices are the ones followed to have a detailed look at the current state and determine future trends or research that can be carried out in this field.

This section presents the results obtained by processing the sample generated following the use of the Web of Science database and the filtering of the previously mentioned scientific works.

The sample includes 261 research works, out of which 236 are scientific articles and 25 are proceedings papers. In Figure 5 is presented the number of articles written each year, over the last 5 years. Since the year 2024 is still in progress, the number of papers is not the final one. Therefore, the sample obtained presents only the number of papers from 2020 including 9 June 2024. Since the data are incomplete for the current year, it is too early to draw conclusions and trends for this one. Even so, it can be remarked that the concern for digital platforms in the immediate period generated by the pandemic is noteworthy. Governments from all over the world proposed immediate safe measures due to social distancing and lockdowns. To support citizens in their relationship with public institutions concerning activities such as taxes, applying for various documents, etc., the phenomenon of digitalization increased. Due to this, the topic of digital public services became one of interest for scientists and researchers to support the development of this area and seek the concerns that should be addressed to improve usage among all citizens. In 2022, the situation was more stable; therefore, the need for digital transformation slightly decreased and the focus moved to other areas that were impacted by the pandemic. By the year 2023, the center of attention may have been taken by increasing digital public services since societies try to become more sustainable and smarter. The interaction between citizens and public state organizations is following a hybrid model so far, and the growth of scientific papers in this field may show new updates and findings as well as outcomes from initial measures taken during the pandemic period. The decrease is because the research and measures proposed in the immediate period of the pandemic started in the following years. Thus, stagnation is normal, followed by growth in the near future to see the effects of the implemented solutions.

Following a co-authorship type of analysis, out of the 789 authors, it was determined a number of 33 who have a minimum of two published papers and two citations for each of them. This means that the authors extended their research to gain more understanding toward citizens’ engagement on using digital public platforms. In

Table 3. Top scientific papers based on citation number.

Author	No. of Documents	No. of Citations	Total Link Strength	Impact Index	Affiliation
Lee, Byung-Kwan	2	79	2	39.5	Hanyang University
Lee-Geiller, Seulki	2	79	2	39.5	Yale University
Johnson, Peter A.	2	59	2	29.5	University of Waterloo
Robinson, Pamela J.	2	59	2	29.5	Toronto Metropolitan University
Faber, Bram	2	37	0	18.5	Rekenkamer Metropool Amsterdam
Brescia, Valerio	2	31	3	15.5	University of Turin
Secinaro, Silvana	2	31	3	15.5	University of Turin
Chohan, Sohail Raza	2	25	2	12.5	Nanjing University
Hu, Guangwei	2	25	2	12.5	Hong Kong Polytechnic University
Edelmann, Noella	2	21	0	10.5	Danube University Krems
Wilson, Alexander	2	21	0	10.5	University of Plymouth
Grossi, Giuseppe	2	20	2	10	Kristianstad University and Nord University

is displayed the scientific research having the highest number of citations, to highlight the relevance of the indicated results and to rank them according to this indicator. The first two positions are occupied by Lee Byung-Kwan and Lee-Geiller Seulki, authors who are affiliated with different universities. Both have an identical number of documents and citations on the topic of digital public services.

An analysis also was carried out where the sources that represent the sample extracted from Web of Science were identified. Thus, a bibliographic coupling and unit of analysis sources analysis was carried out, where 174 sources were identified. From these sources, with a minimum number of four published scientific papers, resulted in 8, which are presented in

Table 4. Distribution of scientific papers by source.

Source	No. of Documents	%	No. of Citations	Impact Index	Total Link Strength
14th International Conference on theory and practice of Electronic Governance (ICEGOV 2021)	4	7.14285714	3	0.75	17
administrative sciences	5	8.92857143	44	8.8	158
Government Information Quarterly	20	35.7142857	444	22.2	431
international Journal of Public Administration	5	8.92857143	39	7.8	147
Proceedings of the 23rd Annual International Conference on Digital Government Research, DGO 2022: Intelligent Technologies, Governments, and Citizens	5	8.92857143	5	1	115
Public Management Review	4	7.14285714	1	0.25	141
Sustainability	8	14.2857143	104	13	123
Transforming Government: People, Process, and Policy	5	8.92857143	10	2	142

. The journals that have indexed the most scientific articles together with citations are Government Information Quarterly (20 articles) and Sustainability (8).

To determine from which countries are researchers who manifested an interest for this topic, a co-authorship type analysis was carried out, and the unit of analysis was given by the countries. This analysis shows the collaboration patterns between authors, institutions, or countries. It highlights key contributors and their connections. The maximum number of countries per document is 25, and the minimum number of documents of a country is 5. Out of the 67 countries, 23 meet the thresholds, Romania being one of them (Figure 6).

Another analysis was made to present the network of citations based on the authors. Here, VOSviewer was used to visualize the networks using citation and as a unit of analysis documents. The minimum number of citations of a document was established to be 3, and out of 261 resulted 120 scientific papers. Most of the items from this network are not connected to each other. The largest set of connected items consists of 12 items. These items are included in five clusters, the largest one consisting of three links.

A co-occurrence analysis of keywords was performed, choosing those that appear at least 5 times in a publication. Of the 1381 keywords, 102 met this criterion. The first cluster includes 36 keywords and is represented by the color red; the second cluster includes 29 words, and it is represented by the color green; the third cluster has 15 keywords represented by the color blue; the fourth cluster has 14 items and is arranged in the color yellow; and the fifth cluster, purple in color, has 8 keywords (Figure 7). The first two clusters comprise 65 of the 102 words. It can be noted that most of the keywords are found in the first two clusters, which associate citizen involvement and digital public services (

Table 5. Top keywords based on number of occurrences.

Keyword	Occurrences	Total Link Strength	Cluster
e-government	82	470	1
engagement	63	366	3
social media	59	331	3
participation	42	208	1
citizen participation	35	209	2
adoption	33	223	4
technology	31	158	2
governance	28	163	1
trust	28	197	4
government	23	139	1
internet	23	129	5
coproduction	21	140	1
e-participation	21	144	2
model	21	115	2
citizen engagement	20	122	2
citizens	20	132	3
impact	20	106	3
civic engagement	18	102	5
transparency	17	125	3

).

The analysis presented in Figure 8 is co-occurrence and has the unit of analysis, the minimum number of occurrences of a keyword, which was established to be 5. Of the 941 keywords, 35 meet the threshold. There are four main areas as follows: 1. Social media: local government, Twitter, Facebook, smart cities, social media, which are digital channels that allow free communication and news posting. Through these channels, public organizations can communicate news to citizens and make their processes and decisions transparent, which leads to citizen engagement and the possibility to acquire their opinions; 2. Public engagement: civic engagement, participation, e-government, e-participation, public engagement, crowdsourcing, artificial intelligence, public participation, citizen engagement. These keywords sustain the main theme of digital public services and is interconnected to all areas as the electronic participation of citizens sustains the development of intelligent platforms by public organizations; 3. Digital platforms: sustainability, digital public services, co-creation, digital transformation, digital platforms, public services, public administration, digital government, which represent the means used by citizens to perform activities and engage with citizens through friendly interfaces; 4. Digital health, big data, technology, trust, transparency, smart cities, citizen science, COVID-19, governance, innovation, citizen participation.

3.2. The Perspective of Romanian Citizens toward Digital Public Services

To see the perceptions of Romanian citizens on the acceptance and use of digital public services, quantitative research was carried out. The answers of the participants in the study were analyzed using the SPSS statistical analysis tool (Version 23, IBM, New York, NY, USA), the questions following the factors that influence the citizens’ intention to use the digital platforms made available by public organizations of the Romanian state. Regarding the respondents of this questionnaire, 42.9% of them are male, and 92.9% of the total have as their environment of origin the urban environment. From the point of view of age (

Table 6. The respondents’ age.

Age Category (Years)	Percent (%)
26–35	85.7
36–45	7.1
46–55	3.6
56–65	3.6
Total	100.0

), the majority are between 26 and 35 years old (85.7%), most having as their last level of education (

Table 7. The last level of education completed.

The Last Level of Education Completed	Percent (%)
High school	3.6
Bachelor’s degree or equivalent level	14.3
Master’s degree or equivalent level	64.3
PhD or equivalent level	17.9
Total	100.0

) the master’s level (65.3%). In Table 5, it can be noted that most of the people who answered this questionnaire also have a high level of education. This aspect can be explained by the fact that people with higher education are often actively involved in solving society’s problems, because education can give them the necessary skills to understand the complexity of these problems. In their activity in such interest groups, these individuals do not limit themselves to superficial comments, and they more easily understand the importance of consulting the voice of the people before adopting measures. Thus, educational experience increases the level of political knowledge (Rasmussen 2016) and the level of civic involvement (Hope and Jagers 2014), it is not surprising that most of the people who answered this questionnaire also have higher education.

Among them, all respondents use a laptop or a computer (100%), 92.9% a smartphone, and 35.7% a tablet, all of which are connected to the Internet for their daily use. In addition, of all respondents, 96.43% have used digital public services at least once; only 89.29% of them intend to use them in the future.

Regarding the types of platforms or mobile applications associated with national public services (Figure 9), the results of this study indicated that the most used platform by the respondents is the www.ghiseul.ro platform (67.9%), followed in percentage terms by the Electronic National SEN system (53.6%), applications related to various public institutions or ministries (35.7%), applications associated with public parking services (35.7%), and applications related to town halls (32.1%).

Regarding the most used digital public services by the respondents (Figure 10), the top is led by the services of payment of fees and taxes (75%) and payment of parking fees (75%). Also important are the payment of utilities (for example, Apa Nova Bucharest—53.6%), payment of fiscal obligations to National Agency for Fiscal Administration—ANAF (28.6%), payment of the counter value of registration plates (25%), services related to the management of car premises/registration certificates/own authorization (25%), paying the passport fee (21.4%), and obtaining the criminal record certificate (21.4%).

The next aspect of research was related to the advantages of using digitized public services (Figure 11).

Thus, the participants in this questionnaire mentioned that the most important advantages of the digitization of public services are represented by:

• Saving time and resources (85.71%)—people no longer must stand in long queues or travel long distances to benefit from public services.
• Increased accessibility (71.42%)—by the fact that they can access and use online services, without having to physically travel to government offices.
• Reduction of bureaucracy (67.85%)—procedures and forms are simplified.
• Supporting sustainable development by eliminating papers (67.85%).
• Increased efficiency (57.14%)—by automating processes and reducing human errors.
• Cost reduction (57.14%)—over time, digitization can lead to significant savings for governments and citizens.
• Reducing corruption (53.57%)—digital processes can be monitored more easily and can create a more transparent environment.
• Increased transparency (50%)—information about government decisions, public spending, and other aspects of the administration can be accessed more easily.

The next aspect analyzed was related to the main disadvantages or problems associated with the digitalization process of public services (Figure 12). According to the answers given by the respondents, the main disadvantages or problems associated with the digitalization process of public services refer to the resistance of citizens to change (32.14%), the need to have the necessary digital knowledge to use digital services (21.42%), the lack of technical support in the use of online services (21.42%), the lack of information/knowledge/culture about digitization (21.42%), the problem of access to digital technologies (14.28%), the lack of universal access to the Internet (14.28%), the technological complexity of the digital solutions offered (14.28%), as well as the depersonalization of services through reduced or nonexistent human contact (14.28%).

The correlations between the main factors analyzed in this quantitative research and the variables that describe the original TAM model are presented in

Table 8. Existing correlations between the variables analyzed in the study.

	PU	PEU	ATU	BIU	TCPA	LCEP	CVATT	RU	DIDA
PU	1	0.297	0.680 **	0.533 **	0.099	0.685 **	0.750 **	−0.403 *	0.185
PEU	0.297	1	0.271	0.577 **	0.555 **	0.377 *	0.231	−0.126	−0.15
ATU	0.680 **	0.271	1	0.641 **	0.212	0.743 **	0.775 **	−0.448 *	0.323
	CSEI	DS	DP	PR	FC	CTT	TA	SI
PU	0.376 *	0.632 **	0.640 **	−0.248	0.047	0.28	−0.661 **	−0.039
PEU	0.575 **	0.359	0.429 *	−0.216	0.378 *	0.183	−0.2	−0.341
ATU	0.399 *	0.492 **	0.531 **	−0.208	0.043	0.187	−0.505 **	0.268

** Correlation is significant at the 0.01 level (2-tailed); * Correlation is significant at the 0.05 level (2-tailed); Perceived Usefulness (PU), Perceived Ease of Use (PEU), Behavioral Intention to Use (BIU), Attitude Toward Using (ATU), Digital Inclusion and Diversity Approach (DIDA), Citizens’ Technological Competencies (CTTC), Social influence (SI), Data Security (DS), Data Privacy (DP), Level of citizen engagement and participation (LCEP), Resistance to use (RU), Citizen satisfaction and experience in interacting with digital public services (CSEI), Perceived Risk (PR), Facilitating Conditions (FC), Technical Accessibility and Digital Infrastructure (TADI), Technology Anxiety (TA), Cultural values and attitudes towards technology (CVTAT), Transparency and communication from public authorities (TCPA).

. To interpret the results, the guide proposed by Evans (1996) will be used, which indicated that for the absolute value of the Pearson coefficient (r) values: 0.80−1.0 “very strong correlation”; 0.60–0.79 “strong correlation”; 0.40–0.59 “moderate correlation”; 0.20–0.39 “weak correlation”; 0.0–0.20 “very weak correlation”. To analyze which are the main factors that influence citizens to adopt and use digital public services, the values of the Pearson coefficients between the tested variables and the Attitude Toward Using (ATU) and the Behavioral Intention to Use (BIU) will be analyzed. Observing the data presented in Table 6, there are no very strong correlations between the analyzed factors and the two previously mentioned variables.

On the one hand, when it comes to Attitude Toward Using (ATU), it can be observed that there are strong positive correlations, statistically significant at a 99% confidence level (p < 0.01), between the mentioned variable and LCEP—The level of involvement and participation of citizens (r = 0.743, p < 0.01) and CVATT—Cultural values and attitudes towards technology (r = 0.775, p < 0.01). The factors that positively and moderately influence ATU are DS—Data Security (r = 0.492, p < 0.01) and DP—Data Privacy (r = 0.531, p < 0.01). Moderate but negative correlations exist between ATU and RU—Resistance to use (r = −0.448, p < 0.01), but also between ATU and TA—Technological anxiety (r = −0.505, p < 0.01). A strong correlation between ATU and CVATT has a positive impact on digital public services as citizens open to technology and cultural values that promote innovation and digitalization are more willing to try new platforms and frequently use them. Also, these cultural values can promote trust in public institutions along with their technological solutions, and citizens may be less reluctant to use digital public services. If cultural values are more traditional, the attitude of citizens toward new technology and adopting digital public services can be lower. Digital education in terms of a positive attitude among cultural values that embrace education and developing digital skills by understanding the benefits of technology in the modern era may increase the adoption of digital public services at a national level.

On the other hand, if we analyze the Behavioral Intention to Use (BIU) of digitized public services, four positive, strong, statistically significant correlations can be observed at a confidence level of 99%. These are between the variable BIU and CVATT—Cultural values and attitudes towards technology (r = 0.703, p < 0.01), CSEI—The satisfaction and previous experience of citizens in interacting with digital public services (r = 0.743, p < 0.01), DS—Data security (r = 0.6092, p < 0.01), and DP—Data confidentiality (r = 0.640, p < 0.01). LCEP—Level of involvement and participation of citizens positively and moderately influences BIU (r = 0.588, p < 0.01), while BIU is negatively and moderately influenced by TA—Technological anxiety (r = −0.410, p < 0.01). To test whether age and the last level of education influence the behavior of using public digital services, the Pearson coefficient values were analyzed (

Table 9. Correlations between age, last level of completed education, and the main variables described by the TAM model.

	PU	PEU	ATU	BIU
Age	−0.267	−0.077	−0.354	−0.428 *
Last level of completed education	0.585 **	0.142	0.277	0.173

** Correlation is significant at the 0.01 level (2-tailed); * Correlation is significant at the 0.05 level (2-tailed); Perceived Usefulness (PU), Perceived Ease of Use (PEU), Behavioral Intention to Use (BIU), Attitude Toward Using (ATU).

). A strong correlation between BIU and CVATT highlights that cultural values and attitudes toward technology have a major role in forming the behavioral intention to use digital public services by Romanian citizens. A positive perception of technology sustained by cultural values that encourage the advancement of technology, innovation in this field, and new solutions will increase the intention of citizens to use digital public services and integrate them into their activities with public institutions. On the other hand, if these cultural values are stricter and more traditionalist, citizens will be more reluctant to use the types of platforms offered by governments. Therefore, cultural values play an important role as they can either facilitate or stop the general use of digital public services.

Thus, it can be seen how between the last level of completed education and the Perceived Usefulness (PU), there is a moderate, positive correlation (r = 0.585, p < 0.01), which shows that education contributes significantly to the awareness of the usefulness and necessity of the digitization process of public services. In addition, a moderate, negative, statistically significant correlation at the 95% confidence level is found between age and Behavioral Intention to Use (BIU). This correlation suggests that young people are the ones who use digitized public services more often, while older ones are still reluctant to use technology or are not yet prepared for technological changes.

To test the relationships between gender, background, electronic devices used, and the variables of the original TAM model, the following were applied: the independent t-test and the ANOVA test, the results of these tests suggesting that the analyzed factors do not influence the attitude toward using digitized public services (Appendix A).

4. Discussion

Following the two perspectives discussed in the results of this work, two directions were identified that require the attention of both researchers and public institutions to jointly start actions that can help increase the level of adoption of digital public services in society. The first discovered directive refers to citizen engagement and participation, while the second refers to e-government and digital transformation.

Regarding the first direction identified, Citizen engagement and participation, citizens’ participation has an impact on the provision of public services by organizations. Using the data generated following the interaction of an online service platform in Beijing, the effects of the conscious involvement of citizens on the responsiveness of public service organizations are analyzed (Jiang and Fan 2024). In this scientific paper are found also factors that influence Romanian citizens to use digital public services. These factors are related to Attitude towards using, The level of involvement and participation of citizens, Behavioral Intention to Use, and The satisfaction and previous experience of citizens in interacting with digital public services. The positive outcomes and benefits bought by using digital public platform positively engage citizens and may lead to active participation. By doing so, the level of citizen participation contributes to the co-creation of value for public organizations and increases the quality of the services provided. Their Behavioral intention to use digital public services is given by their level of trust in these platforms along with the advantages brought. If both are fulfilled, it is most likely for citizens to use digital public platforms in the future. The factor represented by satisfaction in using digital public services is based on the prompt response of public organizations to their requests. Building a constructive relationship between the government and citizens by solving problems and needs leads to their involvement (Contreras-Espinosa and Blanco-M 2020).

Governments are adopting multiple digital platforms to promote and increase citizen participation. Their role in the use of online participation platforms is analyzed by Van den Berg et al. (2020). Here, as well as in the current study was found that disadvantages are related to gaps and access to technology among the population. Knowledge is needed to use digital services, along with access to the Internet and digital technologies. By educating citizens and providing access and support, the level of using digital public services will increase in time. Also, older adults and citizens with lower education levels face difficulties when using technology. These findings were also encountered in the present research, as younger people have more digital skills and knowledge when it comes to using digital platforms; therefore, they are more confident in using digital public services. There are also presented factors that contribute to the engagement of citizens in using digital public services, and some of these factors are also encountered by Romanian citizens as follows: The level of involvement and participation of citizens, as this varies by digital literacy and reachability, Data security, which is found to be the main pilot of citizens trust, Data privacy, as it is linked to inclusivity, Resistance to use is less encountered at young generations that have digital skills, Technological anxiety, which is usually between older people who are more reluctant to use technology, Cultural values and attitudes towards technology may have an impact on participation rates, and Data confidentiality that is highly important to overcome the resistance to use. Sociodemographic variables along with accessible information contribute to e-participation (Secinaro et al. 2022).

Another study in this direction indicates that social networks represent a favorable environment for citizens who want to initiate conversations with the government (Homburg 2022; Ly and Ly 2023; Xu et al. 2022), as well as the conditions in which they can be used. Municipalities that also have a Twitter account and have experience with social networks have increased online involvement (Faber et al. 2020). In this article are also found factors that influence citizens in using digital public services. As well as in the case of Romanian citizens, in Dutch municipalities can be observed the Attitude towards using, which here is determined by political engagement, The level of involvement and participation of citizens is increased in communities that are politically engaged, Data security is linked to usage of public digital platforms, Data privacy is also a key to gain public trust, Behavioral Intention to Use is correlated to the socioeconomic status of citizens, and their Level of involvement is higher for those who are actively engaged in political or social concerns. Using the data extracted from Twitter, the components of citizens’ involvement were identified, in a sample from Italy, namely popularity, virality, and commitment (Blasi et al. 2022). The factors that lead citizens to use social networks are identified through correlation and regression analyses and consist of activity on the social network page, interactivity, and mood (Svirak and Urbánek 2023). In another study, the form of co-production of the basic social networks within the citizen–government relationship is determined in two directions: government–citizen regarding supply and reporting to citizens (8.96%) and communication to the government regarding information and exhortation to citizens (86.62%) (Cho and Melisa 2021). Analyzing 505 posts from social media indicated that a contextual driver for citizen engagement is discretion, as citizens get more involved in decision-making when they feel that municipalities exert an increased discretionary power (Faber 2022).

For an inclusive involvement, lean digital tools and webinars are recommended to governments (Hofstra et al. 2023). Digital media changes the way governments plan and implement their public policies, and these transformations in the rural environment are documented in a parallel between the USA and Romania (Vlad et al. 2023). Using mixed research, it is analyzed on the models of use, experience, and acquisition of the information provided by the government through social media. Users who have high ICT knowledge tend to rely more on digital media for obtaining data (Yavetz and Aharony 2022).

The manifestation of the organizational conditions for co-production in the network was analyzed in the implementation of participatory digital platforms (Kleinhans et al. 2022). It turned out that these conditions are more co-constitutive and co-evolutionary, having strong connections between the compatibility of public agencies, organizational cultures, attitudes toward co-production, and incentives for it.

An analysis of a voting platform in Moscow is presented, determining the profile of a citizen based on the weight of active participation and the degree of involvement depending on the topic (Arkhipova and Starshinova 2021).

In the study of van der Giessen and Bayerl (2022), the structure of technological frameworks is presented, using mixed methods, so that the design of platforms is one that considers the target groups. The provision of intelligent government services leads to a high degree of trust granted by citizens, who can benefit from fast interactions using digital technologies (Hartanti et al. 2021). This concern related to data privacy of digital public services is also found among Romanian citizens. When using this platforms, personal data are needed; therefore, strong security measures must be adopted and frequently updated, to protect the sensitive information.

The research of Dragomir (2021) summarizes the latest trends on how media companies communicate with their audience in the digital economy and identifies methods of engaging citizens in the public media of the future.

In another study, the form of co-production of the basic social networks within the citizen–government relationship is determined in two directions: government–citizen regarding supply and reporting to citizens (8.96%) and communication to the government regarding information and exhortation to citizens (86.62%).

Smart governance and its use by citizens are impacted by information transparency and performance expectation, and not by facilitating conditions and personalization (Hujran et al. 2023). Factors that influence the usage degree at Romanian citizens such as Attitude Toward Using, The level of involvement and participation of citizens, Data security, Data privacy, Resistance to use, Technological anxiety, Behavioral Intention to Use, The satisfaction and previous experience of citizens, Level of involvement and participation of citizens are also found in this study. The citizen’s attitude, Behavioral Intention to Use, and Level of involvement and participation of citizens are positive when they see the efficiency of digital public services and if these are reliable. The efficiency and perceived ease of use also influences the citizen level of participation. The ease of use has a positive influence as it reduces the factors represented by Resistance to use and Technological anxiety. Data privacy is fundamental for acceptance and data security for trust and usage of digital public services by citizens.

As for the second direction, E-government and digital transformation, according to a survey study made on government digital transformation in China, privacy concerns, reachability, and the level of trust in technology influence citizens’ digital participation (Yuan et al. 2023). In this research can also be found the factors Attitude Toward Using, Behavioral Intention to Use, and the satisfaction and previous experience of citizens. It is highlighted that citizens have a positive attitude if the organizational platforms have an interface easy to use and accessible. If they found these platforms to be efficient and trustworthy, their behavior is influenced positively. The interaction and updates of the platforms are contributing to engage citizens into using digital public services and increase their level of satisfaction. Data security was also found to be perceived as a concern in the results of this article’s quantitative research and reachability is one of the main advantages in Romanian citizens’ opinion. If the citizens’ personal data is protected while using public digital platforms, they can perform activities such as payments, completing and submitting forms, and consulting and interacting with local authorities from anywhere, at any time. E-government was adopted to support citizens by streamlining processes using new models of governance, offer transparency, and engage them in decision-making processes (Hashim 2024).

The factors related to Data security and privacy, Resistance to use, and Behavioral intention to use are identified in this research as well. Data privacy is a factor that impacts the willingness of citizens to use digital public services, and Data security is considered to be a key factor when it comes to initiatives of e-government in smart cities. These factors that are related to data protection positively influence the resistance to use and the intention as both protect personal and sensitive data. The success of digital public services is related to the quality of organizations knowledge management and may have a positive impact on engaging citizens (Alvarenga et al. 2020). The factor related to satisfaction of citizens is shaped in this paper by effective management and streamlined digital processes. When digital public services are considered to be beneficial, the factor “Attitude towards using” is positively influenced. If the benefits brought by digital transformation are clear and can be seen by citizens, the factor represented by “Citizens’ level of involvement” is higher. The implementation of digital initiatives is constrained by limited resources in South Africa, and a well-defined strategy may support digital governance to engage stakeholders (Osah and Pade-Khene 2020).

In this research, the authors present that availability of resources along with e-government strategies effectively contribute to a higher Behavioral intention and Attitude toward using digital public services. Infrastructure and support have been affecting Resistance to use. A study was also conducted in Greece, using cognitive theory and TAM to analyze the quality of services provided by e-government from the citizens’ point of view (Patergiannaki and Pollalis 2024). In this research, the factors Attitude Toward Using, Level of involvement and participation of citizens, and Behavioral Intention to Use are also encountered. The accessibility provided by e-government as well as the quality of digital services contribute to these factors, along with demographic factors. E-government is better than autocracies as the political regime type has an impact on the e-government conceptualization and its practices along with its assessment of performance (Bougherra et al. 2023). In this paper are identified the factors related to data security and privacy and cultural values, as these can contribute positively in e-government.

An analysis on co-design and co-delivery of a digital platform is conducted to determine how to use ICT on co-production of public services. It is indicated that there are several aspects to be considered: providing a guide, transparency, capacity building, easy handovers, a continuous evaluation, and sustainability (Leonardi and Not 2022). ICT tools influence their level of participation, and, here, an important factor is also expressed by Cultural Values and attitudes for technology. How citizens engage with public organizations using ICT tools supports the innovation in public service and co-production initiatives. These factors were identified among Romanian citizens as well. Utilization and reglementation of ITC contributes positively to e-participation as reglementation bounds ITC with electronic participation (Adam and Dzang Alhassan 2022). ICT regulations secure data and enhance data privacy. Through them, the factors of data security and privacy can increase, and it positively influences the factor of Behavioral intention to use since information is secured properly.

The process of co-creation in terms of services for e-government with a focus on citizen engagement and the role of government is evaluated to gain insights (Khan and Krishnan 2021). In this paper are found two of the factors that are representing the participation and involvement of citizens. These factors are high if citizens are engaged in the process of co-creation and the communication is easy. Evaluation of public administration websites can contribute to the understanding of the needs of citizens. Initiatives taken by COVID-19 supported the adoption of technologies by governments to solve critical issues (Sarantis et al. 2022).

In this paper are identified three factors that were encountered also among Romanian citizens. Here, Attitude Toward Using is influenced not only by the efficiency of digital public services, but also by the transparency of them. Technological anxiety is found in areas where digital literacy is low and citizens have access issues to technology. Friendly interfaces and accessibility are the main pillars to enhance an increased satisfaction of citizens. Blockchain is a solution that can improve citizen participation in digital politics, through decentralized management of public information (Kassen 2023). Blockchain technology increases trust in digital governance by offering complex mechanisms to ensure the confidentiality of data. Therefore, this technology influences factors such as Data security and Data confidentiality. It offers a high level of transparency that lowers the factor represented by Resistance to use.

Citizens have a role in e-government in each value position transitioning from an entity considered to be external and uses digital services to an entity that is engaged and contributes to the development of policies and offers services (Distel and Lindgren 2023). The factors related to Cultural values and attitudes toward technology and Level of involvement and participation of citizens identified in the study for Romanian citizens are also analyzed in this article. These two factors are significantly influenced if e-government services are in alignment with citizens’ values. The stages of the development of e-government from a public value perspective is analyzed from three perspectives: efficiency, transparency, and engagement (Zhang and Kimathi 2022). From the factors identified among Romanian citizens, only the factors represented by Behavioral intention to use and Level of satisfaction while using digital public services are linked to the perceived value and expectations of citizens from e-government.

The success initiatives for electronic participation are studied to find out common key factors. It was discovered that factors such as superior management support, political leadership, levels of collaboration, and others have an impact on e-participation. Mostly these factors are related to organizational dimensions and processes and not with ITC or institutional context (Royo et al. 2024). Also, factors such as Level of involvement and participation and satisfaction of citizens are met if transparent and responsive digital public services are provided.

In the ecosystem of government big data, there are a lot of challenges and a lack of definitions since it is a fast-evolving area (Shah et al. 2021). A systematic literature review was conducted to delve into these issues and establish a framework to organize and explain these gaps. Concerns about data security and privacy are important in government big data ecosystems. If data are secured properly, the resistance to use e-government platforms can decrease.

5. Conclusions

As a result of the bibliometric research, from the 261 scientific papers that treat the subject of digital public services, the most (81 publications) were published in 2021, recently after the beginning of the COVID-19 pandemic. Several researchers (789) performed analysis in the field of digital public services, but only 33 were more concerned about this topic and their work was considered more valuable judging by the number of their citations and impact index. From the perspective of relevance given by the number of citations, the authors Lee Byung-Kwan (Hanyang University) and Lee-Geiller Seulki (Yale University) have 79 citations each and an impact index of 39.5.

The two main sources that indexed scientific papers on this topic are the Government Information Quarterly, where can be found 20 publications out of 261, and Sustainability, where there are 8 articles. These numbers place the two sources on top of the others that were presented above in Table 4.

Authors from 67 countries conducted research on digital public services, but only 23 countries had more than five papers published, Romania being one of them. The authors from these countries manifested a higher interest on this topic, contributing to the academic and public organization environment by providing scientific results and measures for improvement. Also, only half of the countries are part of the European Union, where is adopted the analysis on a common digitalization index—DESI. A remark here is given by the fact that even if it were determined that 120 papers were cited at least 3 times, they are not connected to each other. This may suggest that each study was individual, and not based on others already published. When developing a new article, it is recommended to conduct thorough research on the current state of the art in order to approach the existing concerns, to cover them by using other methods and to see what the latest discoveries in that area are.

In this sample of 261 research papers, the keywords that appeared most and to which digital public services are associated are e-government, engagement, social media, participation, citizen participation, adoption, and technology. It must be taken into account that the sample had as a basis the following keywords: citizen engagement, digital public services, government online platforms, digital public platforms, and e-government. Citizen engagement has an important role here as this research is focused on the utilization of digital public services by citizens to be able to find the factors that have an impact on the intent to adopt these platforms.

Even if the research evolution on digital public services presents an increased interest in correlation with citizens’ engagement, there is a small number of authors who have been more preoccupied with this topic and of which papers have a strong impact. This suggests limited continuity and also development of sustainable studies. The lack of connectivity between studies may imply a gap of research, which may lead to difficulties in creating a consolidated theoretical database and developing new coordinated research directions. The main journals that have published these scientific articles are important forums that facilitate the collaboration and dissemination of results worldwide. The implication of Romania as one of the countries interested in this topic highlights the concern to develop optimized digital public services at a national level.

The results of the quantitative research indicate that the most important factors that influence the level of adoption of digital public services by the citizens of sector 6 of the city of Bucharest are: LCEP—The level of involvement and participation of citizens, CVATT—Cultural values and attitudes toward technology, DS—Data security, DP—Data privacy, RU—Resistance to use, TA—Technology anxiety, Satisfaction, and previous experience of citizens in interacting with digital public services, LCEP—Level of involvement and participation of citizens, age and education level of citizens.

These results indicate that those who consider that their opinions are important to change something in society and especially are heard by public institutions are more likely to use digitized public services. Likewise, people who believe that the digitization of public services can lead to a more efficient and advanced society, being usually open to trying new ways of communication, tend to also use new technologies within public services. Moreover, if citizens trust that their personal data are protected while using digital public services and are convinced that public authorities have implemented effective measures to prevent unauthorized access to their data, they are more likely to use digital public services in the future, as they are secure and protect their information from cyber threats. In addition, those who believe that their personal information is treated confidentially by the authorities are willing to provide these data also online for the use of digital public services.

The strong correlation between ATU and CVATT as well as BIU and CVATT reveals that the positive perception of citizens toward technology can see the benefits of using it and will more likely use digital public services. The cultural values that promote innovation and digitalization play a key role in forming the behavioral intention of citizens to manipulate these platforms. Where societies have a more conservative attitude, the adoption of digital public services may be slower.

Also, those who show a higher resistance to change are not eager to try digitized public services, preferring to use traditional or conventional methods to interact with public authorities instead of digital public services. Familiarity with old procedures but also technology anxiety contributes moderately to the use of embedded technologies in public services. The uncertainty and fear of possible problems or difficulties that may arise while using digital public services discourage many from using these digitized services.

In addition, citizens whose satisfaction and previous experience in interacting with digital public services are positive declare themselves satisfied with the way digital public services have solved their problems or satisfied their past and present needs, being willing to continue using digital public services to the detriment of classic ones.

In conclusion, to stimulate the utilization of digital public services on a larger scale in Romania, it is important to approach the technical aspects by simplifying the interfaces of digital public services and to create a reachable and friendly technical support. The cultural and educational factors are essential as well as the promotion of digitalization benefits among citizens. By embracing methods to grow digital competencies of the Romanian population, digital public services usage can grow. Only through an integrated approach that takes into consideration benefits, disadvantages, and cultural values can an efficient and sustainable adoption of digital public services by Romanian citizens be assured.

The limitations of bibliometric research are drawn by the dependency between data sources and the selection of articles. Since this research is based on available databases such as Web of Science, articles that are not indexed in these may lead to an incomplete overview in this topic. Furthermore, the lack of connection between scientific papers may limit the capacity to identify well-defined research paths or understand the interactions and various angles in this area. Therefore, more collaborative studies to develop an integrated and complex database are important for the adoption of digital public services among citizens on a larger scale.

The limitations of the quantitative research are given by the proposed sample. Therefore, the results of this research must be treated with care, since the study is a pilot one, being applied to a relatively small sample, where the response rate was small compared to what was wanted to be achieved to be as representative as possible the better of the answers. The generalization of conclusions is little. Also, the understanding of questions or predisposition to answer in a certain way may affect the accuracy of data. In the future, it is desired to update and improve the questionnaire with several factors and items already validated in established studies from the specialized literature, a questionnaire that will then be distributed over a longer period at least at the level of the city of Bucharest.