Using Mixed Methods to Identify Evaluation Indicators for Green Railway Transportation Operations in China

Abstract

The transition of China’s railways from aggressive expansion to sustainable development has sparked widespread discussions on green railways. Previous studies have primarily focused on the fundamental aspects of green rail design and construction. However, the green operation phase, one of the most critical stages in the entire lifecycle of railways, has been overlooked. This study used a mixed-method approach, combining systematic review and qualitative analysis, to identify significant environmental, economic, and social evaluation indicators for green railway transportation operations. Through an examination of 123 articles and interviews with four professors in academia, two Planning and Design Institute technicians, one government staff, and one railway practitioner, this study identified 17 key indicators associated with green operations in railways. The results showed that previous studies in the environmental aspect primarily focused on noise pollution, water pollution, solid waste, ecological conservation, and the use of construction materials. In the social part, the main concern is social equality. Green railway operations have the potential to impact social equity through the movement of people and goods, which makes accessibility a preferable evaluation measure. Regarding economic indicators, the influencing factors are more complex (such as regional GDP per capita), making it difficult to evaluate the effectiveness of green railway operations directly. We suggest the inclusion of more economic-related indicators that directly correlate with green operations in railways.

1. Introduction

China has made remarkable progress in expanding its railway network throughout the country, leading to the second-biggest rail network in the world. The operating mileage of railways has risen from 74,400 km in 2004 to 155,000 km in 2022, with an average increase of 4.19% per year [1]. The existing railway network has fully connected all provinces, covering over 82% of county-level administrative regions and servicing 20.8 billion people annually across the nation [2]. After robust railway network expansion (from 2009 to 2019), the annual growth rate of operating mileage has gradually decelerated since 2019 [2]. Driven by efforts to align with more sustainable development goals (e.g., carbon neutrality), green operations in railways have attracted much attention in recent years [3].

The development of the railway system in China has transitioned from aggressive expansion to a focus on sustainable operations, as shown in Figure 1. During the phase of aggressive expansion, China drew upon the fundamental experiences and lessons learned from railway systems in other countries (e.g., Japan [4], France [5], and Germany [6]). Globally, many countries have adopted strategies of deregulation and privatization for their regional railway companies [7,8,9]. Meanwhile, China Railway implemented a unique organizational structure and became the largest government-owned vertically integrated railway operator [10,11]. Hence, while focusing on sustainable operations in railways, it is insufficient to rely solely on previous studies conducted in other countries [12,13,14]. The Chinese railway system demands a tailored approach to address the specific challenges arising from its substantial market scale and unique organizational structure.

Previous studies have discussed the sustainable development of the Chinese railway system. Green infrastructure and equipment in the railway system have been widely investigated. Wang (2021) proposed a lifecycle management framework to minimize the total management costs when applying intelligent high-speed rail infrastructure [15]. Zhang et al. (2023) developed an energy-saving framework to optimize the total energy consumption of inter-station equipment [16]. Xu et al. (2023) designed energy-saving strategies for railway vehicle maintenance [17]. Some research also explored the process for achieving sustainable development goals in railways. Lui et al. (2017) established the evaluation system for the construction phase of the railway, considering its impact on environmental pollutant reduction and social governance [18]. Cornet et al. (2018) stated how railway infrastructure construction impacted carbon emissions and biodiversity [19]. Liu and Lin (2021) evaluated the carbon reduction achieved through the adoption of electric locomotives in China’s rail sector [20].

While the aforementioned studies designed various evaluation frameworks for the railway system, the existing investigation has focused chiefly on the design and construction phases of railways [18,19,20]. It is of great significance to the development of railways to make effective use of railway resources [21]. There is a notable lack of discussion in previous studies regarding the implementation of green operations in the railway system in China. The operation and maintenance stages contributed 31.60% of carbon emissions and 35.32% of the energy consumption of the entire rail system [22]. Given the imperative of the sustainable development goals [23,24] and the potential adverse impacts associated with the operation phase [22], it is urgent to explore significant evaluation indicators for developing sustainable operations in the railway system in China. To fill this research gap, this study proposes an evaluation framework of green operation railways in China. The structure of this paper is as follows: Section 2 presents a brief review of the existing research on green operations in the railway system. Section 3 determines the scope of green operations in railways. Section 4 introduces the evaluation process and identifies significant factors. This study’s primary results are discussed in Section 5, followed by a conclusion in Section 6.

2. Literature Review

Sustainable transport refers to ways of transportation that are sustainable in their social and environmental impacts. Previous studies mainly focused on the sustainable development of public transport, especially buses and metro systems. Perra et al. (2017) constructed a systematic evaluation system for public transport systems in Thessaloniki, Greece, to determine the level of sustainable transport in Thessaloniki [25]. Chen et al. (2023) took the Beijing subway as an example to evaluate the TOD (Transit-Oriented-Development) level of the existing core subway station area to effectively promote the sustainable development of the subway [26]; Li et al. (2020) evaluated the operational efficiency of buses by analyzing the external environmental factors of buses in order to improve the attractiveness of urban public transport and green travel [27].

Some quantitative studies have evaluated the railway systems’ environmental impacts throughout their lifecycle. Khan et al. (2019) investigated the effect of the construction phase of rail lines on environmental pollutants, such as air and noise pollution, in urban areas [28]. Teng et al. (2022) estimated carbon reductions achieved by trains throughout the implementation of clean energy sources compared to fossil fuels [29]. Kostianaia et al. (2023) highlighted the significant impacts of climate change on railway design and construction [30]. Several research efforts have conducted powerful evaluation indicators to determine the sustainable development of railways. Li et al. (2023) established a safety evaluation model of railway tunnel structures using the fuzzy comprehensive evaluation method and proposed engineering geological conditions and systems [31]. Zhang et al. (2023) identified the performance evaluation indicators of railway transportation, and the relevant indicators of green development include the proportion of an electrified railway, comprehensive energy consumption per unit transportation workload, and revenue rate of transporting freight ton-kilometers [32]. The aforementioned studies mainly focused on the sustainability of railways’ design and construction phases, especially their environmental and economic impacts. However, there is a lack of discussion about the social effects generated by railway operations.

Different evaluation systems for green railways have other research areas, evaluation objectives, stages, and methods, as shown in

Table 1. Comparison between evaluation of green operation in railways and others.

Research Area	Evaluation of Green Operation in Railways	Evaluation of Green Construction in Railways	Evaluation of Green Railway Stations	Evaluation of Green Railway
Similarities	All of them are green evaluations throughout the whole lifecycle of railways. They are establishing evaluation systems to consider environmental impacts, such as resource conservation and environmental protection.
Evaluation Object	Operation activities	Construction activities	Station (e.g., equipment and infrastructure)	The whole lifecycle
Evaluation Stage	Operation	Construction	Construction of railway stations	The whole lifecycle
Evaluation Method	Phased-based evaluation	Phased-based evaluation	Object-oriented evaluation	Holistic evaluation
References	[21]	[18,33,42]	[36,37,43]	[38,39,40,41]

. Evaluation indicators for green railway construction have been widely investigated [18,33,34]. Furthermore, railways greatly impact people’s lives, such as environmental and economic aspects [35]. Due to different evaluation stages, they cannot be applied for green operations in railways. The object-oriented evaluation method (i.e., evaluation of green railway stations) only focused on the lifecycle of a specific railway station [36,37]. Operations in railways include more than one object, such as infrastructure, vehicles, lines, and equipment [38]. The lifecycle assessment (LCA) has been widely used to evaluate the whole railway system, aiming to enhance its overall sustainable level [39]. LCA mostly collected a substantial number of categories and indicators, which is limited by the absence of data and homogeneous assumptions [40,41]. Although the evaluation frameworks for different objects or orientations have been widely discussed, the evaluation of green operation in railways remains unclear.

The commonly used methods of constructing evaluation indicators for unstructured problems include literature research, the Driver–Pressure–State–Impact–Response (DPSIR) model, and qualitative research. Xu et al. (2021) built an evaluation system for urban transport based on the literature research method. They verified the proposed design by urban transport systems in Beijing and Chongqing in China [44]. Ding et al. (2015) used the DPSIR and ANP-fuzzy comprehensive evaluation models to develop an evaluation system for the Beijing–Shanghai high-speed railway [42]. Wikstrøm et al. (2020) employed a qualitative research approach to explore promoting decision makers’ views and experiences in the story, supervision, design, commissioning, and implementation of environmental interventions to promote active travel infrastructure [45].

Based on the fact that the above-mentioned green railway operation is currently a non-structural problem and is closely related to other links in railway transportation, the conceptual boundary is blurred, and the research object is novel, so it is necessary to consult experts in multiple fields for advice. Grounded theory, one of the more advanced methods in qualitative research, can collect multiple views by rooting in fundamental data and expert interview data, and the literature research method can reduce the limitations caused by the subjectivity of qualitative research. In the above literature, it can be found that there are some gaps in the research on green railway management evaluation. Finally, this study conducted four steps: summarizing the concept of green railway management through literature research, constructing a preliminary indicator database, using qualitative research methods to improve the indicator system further, and establishing a theoretical approach to green railway operations.

3. Green Operation in Railway

As shown in

Table 2. Summary of policy documents related to green development by various organizations.

Source	Publication	Year	Keyword
The United Nations (UN)	China Human Development Report 2002: Making Green Development a Choice	2002	Reduce pollution and protect the environment
The United Nations Economic and Social Commission for Asia and the Pacific	Ministerial conference on environment and development	2005	Low-carbon environmental threats and ecological scarcity
Organization for Economic Co-operation and Development (OECD)	Towards Green Growth	2011	Resource efficiency, natural assets, and environmental services
The World Bank	Inclusive Green Growth: The Road to Sustainable Development	2012	Efficient, clean, and resilient
The Fifth Plenary Session of the 18th Central Committee of the Communist Party of China	The Central Committee’s Proposal for Formulating the 13th Five-Year Plan for National Economic and Social Development	2016	Environmental capacity, resource capacity, and environmental protection

, previous policy documents related to green development focused on the integration and harmony of ecosystems and economic systems. Resource conservation and environmental protection have become widely recognized as crucial aspects of green development. This recognition was reflected in various publications and reports, such as the United Nations’ “China Human Development Report 2002: Making Green Development a Choice” in 2002 [46], the United Nations Economic and Social Commission for Asia and the Pacific’s ministerial meeting on environment and development in 2005 [47], and the OECD’s book “Towards Green Growth” in 2011 [48], which strongly emphasized the keywords related to environmental protection and resource conservation. In 2016, during the Fifth Plenary Session of the 18th Central Committee of the Communist Party of China, the “Proposal for Formulating the 13th Five-Year Plan for National Economic and Social Development” was issued [49]. This proposal highlighted that green development represents an innovative model building upon traditional development and acts as a new development paradigm based on the constraints of ecological environment capacity and resource-carrying capacity. Additionally, environmental protection was recognized as a crucial pillar for achieving sustainable development.

As theoretical research on green development has progressed, it has become apparent that the consensus on resource conservation and environmental protection alone is insufficient to address the current context in China. Wu et al. (2017) proposed three core elements of “environment, economy, and society” for green development, which aims to protect ecological benefits while ensuring economic and social benefits [50]. Zhao et al. (2017) expanded the concept of green development, highlighting the symbiotic relationship between natural, economic, and social systems [51]. Additionally, Qian et al. (2020) emphasized that green development prioritizes ecological considerations while calling for a more systematic, holistic, and coordinated relationship between economic, social, and natural systems [52]. Previous research suggested that future research should explore the inter-relationships among environment, economy, and societal aspects to advance the understanding of green development.

In the context of railway operation, it often overlooks the economic flow and the societal impacts brought by accessibility [40], which is based on the movement of people and logistics facilitated by transportation effects. Green development in the railway sector goes beyond promoting a low-carbon transportation mode and encompasses the railway’s contribution to environmental protection and resource conservation within the broader transportation system. Moreover, it can potentially exert indirect impacts on the social and economic aspects [40,41], leading to changes in their characteristics resulting from railway operations, such as helping products sell from backward areas. Ultimately, the green effect of railway operation is reflected in the integration of the environment, economy, and society, as shown in Figure 2.

4. Materials and Methods

This study used a mixed-method approach, combining systematic review and qualitative study, to identify significant evaluation indicators. The proposed evaluation framework includes two steps: a systematic review for extracting important evaluation indicators from previous relevant studies (as shown in Figure 3a) and qualitative research for identifying significant evaluation indicators (as shown in Figure 3b). In Figure 3a, this study searched and screened relevant articles through keywords from the dataset of CNKI (China National Knowledge Infrastructure), spanning from 1998 to 2021, assessed the reliability of participating staff, analyzed the collected indicators, and established the initial indicator dataset. Grounded theory’s primary goal is to develop an approach based on actual evidence. With direct observation, researchers synthesize experience from the original data before moving on to the theoretical level to develop a theory, which is the core idea underlying the nature of occurrences as reflected by methodically gathering data signals. Based on the grounded theory method, this research further studies the results of the systematic review, and the specific flow is shown in Figure 3b. We invited 8 relevant interviewers (i.e., 4 professors in academia, 2 technicians in the Planning and Design Institute, 1 government staff, and 1 railway practitioner) to conduct in-depth interviews as part of the qualitative study. This study further analyzed the statements of all interviewers and identified significant indicators.

4.1. Systematic Review Procedure

A systematic review has been shown in this study to examine the evaluation system for green operation in railways. As one of China’s authoritative research databases, the dataset “CNKI” encompasses various documents, including research articles, yearbooks, standards, scientific and technological reports, government documents, laws and regulations, and other diversified information materials. This study selected the CNKI dataset as the primary data source. To simplify the systematic review process, all the various types of documents within the CNKI dataset are referred to as “articles” in this section. Following the routine review procedure in Figure 3a, this study collected and analyzed articles in relevant domains (i.e., transport, infrastructure, and railways) from the CNKI dataset, spanning from 1998 to 2021. The query is (“Sustainable Transportation” OR “Green Transportation” OR “Green Railway” OR “Railway Operation) AND (“evaluation framework” OR “evaluation index”), as shown in Figure 4. After conducting the initial search of 8180 articles, the addition of the keyword “evaluation” reduced the dataset to 791 articles. Following the screening of titles, 337 articles remained, and after reading the abstracts, 33 articles were excluded. Following the aforementioned procedure to narrow the scope and ensure relevance to the evaluation of green operation in railways, 123 papers were selected for further in-depth reading, as indicated in Figure 4. The literature selection of different topics is shown in

Table 3. Number of articles during the systematic review procedure.

	Number of Articles
	Sustainable Transportation	Green Transportation	Green Railway	Railway Operation
Initial searching	1958	41,760	515	1429
Manual screening associated with “Evaluation”	242	349	118	84
Preliminary screened results	60	37	23	5

.

4.1.1. Reliability of Dataset

In Figure 3a, 2 staff associated with railways manually input the data source. To measure the inter-staff reliability, this study used the kappa coefficient ($k$) (as shown in Equations (1)–(2)). Different values of the kappa coefficient showed different levels of consistency: (1) 0.0 to 0.20: very low; (2) 0.21 to 0.40: low; (3) 0.41 to 0.60: medium; (4) 0.61 to 0.80: high; and (5) 0.81 to 1: very high. In this study, the inter-staff reliability is considered acceptable when the kappa coefficient value is more extensive than 0.40. For example, the kappa coefficients for three articles were computed and found to be 0.56, 0.56, and 0.52, respectively. These coefficients indicate a moderate level of consistency across the three articles. Therefore, the two inter-staff reliability factors have credibility in their perception of the indicators.

$$k=\frac{p_o-p_e}{1-p_e}$$

(1)

$$p_e=\frac{{\mathrm{a}}_1\times {\mathrm{b}}_1+{\mathrm{a}}_2\times {\mathrm{b}}_2+\cdots +{\mathrm{a}}_C\times {\mathrm{b}}_C}{\mathrm{n}\times \mathrm{n}}$$

(2)

Here, $p_o$ is the accuracy of the prediction, which can also be understood as the consistency of the prophecy $p_e$ being the contingent concordance; $n$ is the total number of samples; for $k$ categories $\left(k=1, 2, 3,\cdots ,C\right)$, $a_k$ is the number of observed samples for $k$ categories $\left(k=1, 2, 3,\cdots ,C\right)$; and $b_k$ is the number of predicted samples of $k$ categories $\left(k=1, 2, 3,\cdots ,C\right)$.

4.1.2. Indicator Preprocessing

To select significant indicators for evaluating green operation in railways, this study designed a three-step indicator preprocessing procedure (i.e., initial preprocessing, selection by the time-varying frequency pattern, merging indicators with the same function, and modification name), as shown in Figure 5. Firstly, this study extracted 171 indicators from the mentioned 123 articles and then screened for possible duplicates since they were gathered manually. Secondly, we evaluated the frequency of each hand that appeared in the set of selected papers and established the frequency matrix of each indicator $F$ as shown in Equation (3). The element $f_{ij}$ represents the number of times that indicator $i$ appears in the published articles in the year $j$. This study investigates the time-varying pattern of their frequencies with three types: stable, growing, and declining [53]. We classified 62 indicators with regular patterns, 32 with increasing patterns, and 7 with declining patterns. As the importance of the declining indicators has decreased in recent literature, this study eliminated these 7 indicators for further analysis. Thirdly, this study combined duplicated indicators owing to their repeated functions. For example, for the discharge of wastewater and discharge per unit mile of sewage, the units are different, but the orientation of the indicators is the same. Therefore, this study eventually selected 21 significant indicators.

$$F=\left[\begin{array}{ccc}{f}_{11}& \cdots & f_{1J}\\ ⋮& \ddots & ⋮\\ f_{I1}& \cdots & f_{IJ}\end{array}\right]$$

(3)

Here, $f_{ij}$ is the number of times that indicator $i$ ($i=1,\cdots ,171$) appears in the published articles in the year $j$ ($j=1998,\cdots ,2021$).

While implementing the first step, “initial preprocessing”, some data were contaminated due to manual entry errors. This study deleted duplicated data and errors through inspection. The second step is to discover the time-varying frequency pattern. The indicators with low frequency and declining trends are deleted based on their frequency and time-varying trends in previous literature. The third part merges and concludes indicators with the same function. As some indicators have different names but the same definitions, this study deleted them to avoid repeated evaluation.

4.1.3. Identifying Significant Indicators

As depicted in Figure 3b, we conducted a qualitative study to explore the significance of evaluation indicators on green operations in railways. This study invited 4 professors in academia, 2 Planning and Design Institute technicians, 1 government staff, and 1 railway practitioner. Among them, 62.5% are males and 37.5% are females; 50% hold a doctoral degree and 50% hold a master’s degree; and 87.5% have research experience in railway domain. The demographic information, education level, career, and interview types are summarized in

Table 4. Information of invited interviewers.

No.	Gender	Research Direction	Education Level	Career	Interview
1	Male	Railway Operation	Ph.D.	Associate Professor	Face-to-face
2	Female	Transportation Economics	Ph.D.	Professor	Face-to-face
3	Male	Rail and Metro Transportation	Ph.D.	Professor	Face-to-face
4	Female	Rail and Metro Transportation	MEng	Planner	Face-to-face
5	Female	Rail and Metro Transportation	MEng	Planner	Face-to-face
6	Male	Transportation Economics	Ph.D.	Associate Professor	Telephone
7	Male	Transportation Management	BEng	Government Staff	Telephone
8	Male	Railway Operation	MEng	Manager in the railway operation company	Telephone

.

Following the interviews with the interviewers, we manually encoded each sentence, eliminated vague ideas, extracted main ideas, and summarized their perspectives on each evaluation indicator. The final dataset includes 60 initial sets of ideas classified into 11 categories (i.e., energy consumption A11, water consumption A12, sewage treatment A13, solid waste treatment A14, noise and vibration A15, carbon emission reduction A16, clean energy use A17, satisfaction A21, sharing rate A22, accessibility A31, transport volume A32). The interview questions are shown in

Table A2. List of interview questions.

Serial Number	Problems and Processes	Function
1	In the opening remarks, the interviewee was introduced to the connotation of railway green operation, the meaning and content of the indicators, etc., and the basic information the interviewee was asked	Present the research and get to know the respondents.
2	Is your current work related to green railways, green transportation, or sustainable transportation? If so, when and what kind of opportunity will it take?	Elicit impressions
3	Can you talk about your understanding of what green and sustainable are?	Introducing the green theme
4	What are the unique characteristics of green railways compared to ordinary railways? What is the central aspect of “green” in green railways? Or what should it look like?	Ask about green railways.
5	What are the main aspects of railway green operation? In other words, what strategies and measures should be adopted to carry out green railway operations?	Ask about your understanding of green railway management.
6	In terms of the concept of green railways, is there a correlation between the construction and operation phases? If so, how can the whole life cycle be optimized?	Enquire further
7	Thanks to the interviewees, the visit was concluded	Appreciation

Note: this table illustrates the questions for the in-depth interview.

.

5. Results and Discussions

From 1998 to 2021, there are 171 indicators related to the evaluation of green operations in railways in total, including 65 environmental-related indicators (e.g., water consumption and air pollution), 43 economic-related indicators (e.g., the contribution of the transportation sector to GDP), and 63 social-related indicators (e.g., accessibility and employment rate). The list of these 171 indicators and the frequency matrix $F$ are shown in Supplementary Materials. Based on the frequency matrix of each indicator ($F$), this study found that specific indicators that are irrelevant or insignificant still have high frequencies, such as 2 indicators with fuzzy boundaries, 13 indicators with poor independence, and 26 indicators with low correlation associated with railway operation. As for indicators with fuzzy boundaries, such as the level of modernization in publicity and traffic management, they can be divided into two categories: the ambiguity of the application scope (i.e., the content of application of the indicator) and the opacity of the indicator measurement (i.e., whether the indicator is difficult to measure or collect), such as the level of modernization in publicity and traffic management. As for indicators with poor independence (i.e., regional GDP per capita), the relationship between economic indicators and the green operation of railways is not direct but through the flow of people and logistics to improve economic mobility, making the influencing factors complex. As the impact of transportation on the economy is directly generated through accessibility (i.e., the movement of people and goods), regional GDP per capita and accessibility duplicate the movement of people. Indicators with low correlation associated with railway operation are not related to the evaluated object, which cannot effectively reflect the impacts of their evaluation characteristics on the research subject (e.g., rail transit length changes). Hence, this study eliminated these 41 ambiguous indicators.

The qualitative study process is illustrated in Figure 3b. In this study, the in-depth interviews involved eight relevant interviewers, including four professors in academia, two Planning and Design Institute technicians, one government staff, and one railway practitioner. In response to different indicators, there were 60 replies from 8 participated interviewers. The collected data underwent open coding, where conceptualizing labels were assigned to any content recorded in the original data that can be encoded. The open coding process generated 24 concepts (e.g., transportation energy consumption, energy-saving design) and 11 conceptual categories (e.g., energy consumption A11) among discussed indicators, as shown in

Table 5. Result of the qualitative study.

Principal Category	Categorization	Conceptualization
Environment A1	Energy consumption A11	Transportation energy consumption, energy-saving design, energy consumption sources
	Water consumption A12	Sources of water resources consumption, management of water resources consumption
	Sewage treatment A13	Sewage treatment and utilization
	Solid waste treatment A14	Solid waste impact, solid waste treatment
	Noise vibration A15	Sound pollution, vibration effects
	Carbon reduction A16	Significance and source of carbon emission reduction
	Clean energy use A17	Lighting equipment energy, transportation clean energy utilization ratio
Economy A2	Satisfaction A21	Safe operation, quality of service, perfect rules, transportation distance
	Share rate A22	Market share
Society A3	Reachability A31	Industry-driven, convenient travel, resource introduction
	Transport volume A32	Input–output, line capacity

. Through spindle coding, the main links of conceptual types generated by open coding were discussed, identified, and established to express the main links across different parts of data, and based on the three proposed elements of environment, economy, and society, the main category has been classified into three parts. The categorization results of qualitative research aligned with the results in the systematic literature review in Figure 3a. In the qualitative study, according to the categorization results, interviewers’ recommendations, and indicators’ practicability, four indicators were added, three were modified, and seven were merged, as shown in Supplementary Materials.

The theoretical model of green railway operation based on selective coding is shown in Figure 6, where it is shown as a house shape composed of three aspects: environment, economy, and society. Environmental-related factors showed direct impacts on green operation in railways, such as saving sources (e.g., water, ground, material, and energy) and reducing waste/pollutants (e.g., noise reduction and carbon conservation). Economic-related factors provided benefits to the railway industry. For example, the railway market shares directly reflected the percentage of market share generated by the railway industry in the whole transportation system. Revised: The satisfaction of passengers increases the likelihood of choosing the railway as their preferred mode of transportation. Social-related factors enhance the impact of the railway on the economy, such as accessibility and passenger capacity. To sum up, from three aspects, the three sections of green railway management are indispensable in realizing a green railway and achieving business objectives. The contribution of this study to the green railway management theory is shown in Figure 6. Firstly, it provides an overview of the primary direction of environmental impacts on railway operations, including noise pollution, solid waste and sewage, water and energy use, and contaminants. Compared with the construction phase, the operation phase excluded preserving land and building materials, as all buildings have been constructed. Secondly, the movement of people and goods showed a more significant impact on the economy than transportation services. As the underlying mechanism of how railway operations boost the economy is not well defined, it is suggested to quantify the significance of the movement of people and goods rather than the provision of essential transport services. Thirdly, it broadens the scope of the green idea in railway operations and adds a social dimension. There is an agreement about resource conservation and environmental protection in green railway evaluation. Residents’ satisfaction and the railway’s contribution to social equality can improve people’s preferences for the railway, which helps the railway align with more sustainable development goals.

This study eliminated 72 indicators with a less-than-four frequency to determine significant indicators. The frequency of indicators demonstrates the trend of research hotpots related to green operations in railways. If the frequency of an indicator is not sufficiently high, it indicates that the indicator may not be sufficiently necessary. Based on the frequency matrix $F$ in Supplementary Materials, the median and average of $f_{ij}$ are 3 and 5.7, respectively. There are 80.6% of indicators that have a larger-than-four frequency. Therefore, this study considers the 72 indicators with a frequency of less than four insignificant, resulting in 72 remaining in the dataset.

This study compared their time-varying frequency patterns to further screen out the most critical indicators. Due to the insignificance of declining indicators, this study excluded them from the dataset. Meanwhile, some merged indicators exhibit different nomenclatures while conveying the same underlying meaning. Finally, this study identified 20 significant indicators, as shown in

Table A1. Initial library of indicators.

Serial Number	The Name of the Indicators	Unit	Indicator Tendency	Qualitative/Quantitative	Indicator Sources
B1	Transportation cost coordination factor	%	Negative	Quantitative	2, 3, 5, 54, 55
B2	The average distance traveled by passengers	km	Positive	Quantitative	6
B3	Rail passenger share rate	%	Positive	Quantitative	15, 17
B4	Energy consumption per unit mile per capita	person-time·km/kJ	Negative	Quantitative	37, 39, 40, 42, 98
B5	Passenger satisfaction	/	Positive	Qualitative	74, 93, 94, 95, 96
B6	Output value per unit of transport volume (freight)	yuan/t	Positive	Quantitative	83, 87, 89
B7	Passenger traffic turnover	person-times	Positive	Quantitative	85, 88
B8	Average annual safe mileage	km	Positive	Quantitative	139, 140, 141, 142
B9	Speed of railway operation	km/h	Positive	Quantitative	145
B10	Average driving delays	min	Negative	Quantitative	146
B11	Input–output ratio	%	Positive	Quantitative	143, 147, 149, 150
B12	Vehicle occupancy rate	%	Negative	Quantitative	154
B13	Noise equivalent sound level	.db	Negative	Quantitative	155
B14	Equivalent vibration level	%	Positive	Quantitative	156, 158
B15	Proportion of clean energy utilization	%	Positive	Quantitative	35, 163
B16	Total asset turnover	%	Positive	Quantitative	169, 170, 171
B17	Water consumption per capita	t/person-time	Negative	Quantitative	47, 48
B18	Sewage treatment and utilization	/	Positive	Qualitative	46, 49, 50, 106
B19	Passenger supply and demand ratio	%	Positive	Quantitative	20
B20	Solid waste emissions per unit mile transported	t/km	Negative	Quantitative	105, 107

Note: this table is used to illustrate the results of the systematic review procedure.

.

In

Table 6. The dataset of indicators.

	Level 1 Indicators	Secondary Indicators	Indicator Content	Role in Stage	Indicator Tendency	Qualitative/Quantitative	Corresponding to the Results of Qualitative Studies	Corresponding to the Results of the Literature
Evaluation of green operation in railway	Environment (C1)	Energy consumption per ton-kilometer C11	Trains consume an average amount of energy per ton of cargo transported.	Freight	Negative	Quantitative	Energy consumption A11	\
		Energy consumption per person-kilometer C12	The average energy consumption per passenger car transported by the train.	Passenger transportation	Negative	Quantitative	Energy consumption A11	B4
		Water consumption per capita C13	Water consumed during railway operations.	Passenger transportation, freight	Negative	Quantitative	Water consumption A12	B17
		Noise equivalent sound level C14	The noise impact caused by the running train is related to the logarithm of the train.	Passenger transportation, freight	Negative	Quantitative	Noise vibration A15	B13
		Equivalent vibration level C15	The vibration effect caused by the running train is related to the logarithm of the train.	Passenger transportation, freight	Positive	Quantitative	Noise vibration A15	B14
		Clean energy utilization ratio C16	The proportion of clean energy used in the process of railway operation, including photovoltaic, hydropower, wind power, etc.	Passenger transportation, freight	Positive	Quantitative	Clean energy use A17	B15
		Wastewater treatment and utilization C17	Qualitative judgment on the treatment and utilization of sewage in the process of railway operation.	Passenger transportation, freight	Positive	Qualitative	Sewage treatment A13	B18
		Solid waste emissions per unit mile of transportation C18	The average amount of solid waste discharged per kilometer during train operation.	Passenger transportation	Negative	Quantitative	Solid waste treatment A14	B20
		Carbon emission reduction per person-kilometer C19	Compared to the road, a common way to transport goods; each ton of cargo transported reduces carbon emissions.	Freight	Positive	Quantitative	Carbon reduction A16	\
		Ton-kilometer carbon emission reduction C10	Share of rail transport in the integrated transport passenger transport system.	Passenger transportation	Positive	Quantitative	Carbon reduction A16	\
	Economy(C2)	Rail passenger share rate C21	Share of rail transport in integrated transport freight systems.	Freight	Positive	Quantitative	Share rate A22	B3
		Rail freight share rate C22	The degree of satisfaction of passengers or cargo owners with the transportation service.	Passenger transportation, freight	Positive	Qualitative	Share rate A22	\
		Transport service satisfaction C23	The volume of goods transported by train.	Freight	Positive	Quantitative	Satisfaction A21	B5, B8, B9, B10
	Society (C3)	Freight turnover C31	Passenger traffic transported by train.	Passenger transportation	Positive	Quantitative	Transport volume A32	B6, B11
		Passenger transport turnover C32	After the opening of the railway, the difference between the transportation time between the two places and the traditional mode of transportation.	Passenger transportation	Positive	Quantitative	Transport volume A32	B7, B12, B19
		Temporal accessibility C33	After the opening of the railway, the difference between the transportation costs of the two places and the traditional mode of transportation.	Freight	Positive	Quantitative	Reachability A31	B2
		Cost accessibility C34	Trains consume an average amount of energy per ton of cargo transported.	Freight	Negative	Quantitative	Reachability A31	B1, B16

, this study selected 17 evaluation indicators, including 10 indicators at the environmental level, 3 at the economic status level, and 4 at the social level, with quantitative indicators accounting for 94.1% of the overall indicators and positive indicators accounting for 70.5%. The qualitative study further investigated the results of the literature research. The green railway operation evaluation system determines three first-level indicators, namely environment (C1), economy (C2), and society (C3), of which 10 are second-level indicators for the environment, 3 are for the economy, and 4 are for society.

In the secondary indicator layer of the environment, the conceptual categories obtained by qualitative research are energy consumption (A11), water consumption (A12), sewage treatment (A13), solid waste treatment (A14), noise and vibration (A15), carbon emission reduction (A16), and clean energy use (A17). In energy consumption (A11), the energy consumption in transportation is described, and the literature research result B4 is taken; due to the lack of freight-related indicators in the literature research, ton-kilometer carbon emission reduction (C10) is added; B17 indicator of water resource consumption (A12) extraction literature study; literature study on the use of B18 indicator in wastewater treatment (A13); B20 indicator of solid waste treatment (A14) extraction literature; noise and vibration (A15) literature research on B13 and B14 indicators; carbon emission reduction (A16) has no corresponding indicator in the literature research results, so it is added to the indicator system, and the corresponding indicators are carbon emission reduction per person-kilometer (C19) and ton-kilometer carbon emission reduction (C10). In the secondary indicator layer of the economy, the conceptual categories obtained by qualitative research are satisfaction (A21) and sharing rate (A22). Satisfaction (A21) combined the indicator literature studies B5, B8, B9, and B10; the sharing rate (A22) is a literature study of B3 indicators. In the secondary indicator layer of society, the conceptual categories obtained by qualitative research are accessibility (A31) and transportation volume (A32). Accessibility (A31) was modified, and the literature was used to study B1 and B2 indicators. The transport volume (A32) is taken from B6, B7, and B12 and is modified from the perspective of passenger and freight transportation. The frequency of the final indicators has been shown in Figure 7.

This study identified significant evaluation indicators for green operation in railways through a systematic literature review and qualitative research. The association between systematic review and qualitative study is shown in Figure 8. The final results are shown in Table 6, with 10 environmental indicators, 3 economic indicators, and 4 social indicators. The results showed that the environmental aspect primarily focuses on noise pollution, water pollution, solid waste, ecological conservation, and construction materials. In their study on the environmental impact of railway transportation systems, Kollő et al. (2015) mentioned noise emission, energy efficiency, mix of electricity, CO₂ emissions, and other issues. It was designed mainly for environmental factors [35]. Tian et al. (2023) put forward factors such as clean energy use and carbon emission reduction, which are involved in the environmental indicators proposed in the study of green railway construction [54]. Economic indicators are succinct and independent in this research. Regarding economic indicators, the influencing factors are more complex (such as regional GDP per capita), making it difficult to evaluate the effectiveness of green railway operations directly. Chen et al. (2022) pointed out the intricacy of economic influence on traffic in their study [55]. Yang et al. (2021) looked at the relationship between railway traffic and the economy but did not offer particular metrics [56]. Green railway operations have the potential to impact social equity through the movement of people and goods, which makes temporal accessibility (C33) and cost accessibility (C34) preferable evaluation measures that make social indicators creative. Qin et al. (2023) covered the social, economic, and environmental elements of green railways from a comprehensive, macro viewpoint, causing them not to concentrate on any particular stage of the railway lifecycle [24]. Chang et al. (2017) presented pertinent social and economic variables and noted the importance of railroad transportation for social fairness. The study focused on the railway construction phase and did not consider railway operation [57].

6. Conclusions

Against China’s transition from aggressive expansion to sustainable development in railways, research on green railway operations has attracted much attention. Previous studies on green operations in railways primarily investigated the green consensus on environmental protection and resource conservation but overlooked the social impacts (e.g., social equality). To align with the sustainable development in railway operations, this study established an evaluation framework with three directions: environmental, economic, and social. This study used a mixed-method approach, combining systematic and qualitative reviews, to identify significant evaluation indicators.

The proposed mixed-method approach includes a systematic review for extracting significant evaluation indicators from previous relevant studies (as shown in Figure 3a) and a qualitative study for identifying significant evaluation indicators (as shown in Figure 3b). Based on screening titles, abstracts, and in-depth reading, 123 articles were selected from the dataset of CNKI (China National Knowledge Infrastructure). A total of 171 indicators and 980 data points were extracted. Through initial preprocessing analysis (eliminating 41 indicators), frequency analysis (eliminating 72 indicators), and trend analysis (eliminating 7 indicators), 30 duplicate evaluation indicators were merged to form an initial indicator pool consisting of 20 indicators. In-depth interviews were conducted with eight interviewers. We further analyzed the interview data through open, selective, and axial coding to refine the 20 indicators. Eventually, a final indicator system was established, including 17 indicators.

It can be found that the environmental aspect primarily focuses on noise pollution, water pollution, solid waste, ecological conservation, and the use of construction materials. Regarding economic indicators, the influencing factors are more complex (such as regional GDP per capita), making it difficult to evaluate the effectiveness of green railway operations directly. In the social aspect, the main concern is social equality. Green railway operations have the potential to impact social equity through the movement of people and goods, which makes accessibility a preferable evaluation measure.

Research on the literature is used to construct the initial indicators, which are subsequently refined through qualitative research. There are benefits when the indicator is created collaboratively using expert knowledge and literature. Economic indicators are straightforward and independent, but environmental indicators encompass a wide range, covering the majority of resource consumption and pollutants in the business process. In proposing the indicators, the indicators with low independence are disregarded since the mechanism of railway operation on the economy is evident. Innovative social indicators give less consideration to social equity when evaluating railroads. This essay presents a viewpoint on how railroads affect social fairness, providing a theoretical framework for future investigations into the mechanism at play.

In this study, there are certain limitations regarding the selection of articles from the dataset of CNKI, as it focused on China Railway as the research object. While the CNKI dataset is widely utilized in China, it may not encompass the complete scope of research on green railway operations in other countries. Therefore, if further research intends to investigate the green operation of railways in different nations, it is crucial to consider the specific green development policies of local railway systems and determine relevant indicators based on the unique circumstances of each region. Additionally, this study proposes the critical factors of green railway operation rather than a comprehensive factor dataset. They can be further studied in three aspects: environmental, social, and economic in the follow-up research. At the economic level, the influencing factors of the indicators need more discussion. The operation of railways influences the economy through the movement of goods and passengers, and the factors impacting these indicators can be complex and diverse. To comprehensively understand and explain the relationship between economic indicators and green railway management, it is necessary to collect more comprehensive data that provide sufficient support for analysis and interpretation.