Framework of Aspects for the Evaluation of Multimodal Journey Planners

Abstract

Nowadays, several journey planners are available for users with various functionalities. The aim of this research was to provide an overview of European journey planners considering the perspective of the user. Therefore, a framework of aspects was defined, which contains aspects of route planning services, booking and payment, handled data, and supplementary information. Based on these aspects, an evaluation method was elaborated to compare and rank the journey planners. The method consists of two main steps. First, the journey planners were compared (scoring) to each other, which resulted in the general evaluation number. In the scoring step, multi criteria analysis was adapted, because it produces clear and comparable results. As a second step different user groups were introduced and, using preferences of these groups (weighting), the average evaluation number was calculated. Finally, the elaborated method presented a quantified evaluation and ranking of multimodal journey planners, which supports choosing suitable journey planners for the users.

1. Introduction

Travel information can influence the decisions of the passengers, who want to reach their destinations in the most favorable way according to their personal preferences. Travelers need user friendly, multimodal information systems, which can provide information before and during the trip considering combined transportation modes. Journey planners should not only be considered as tools for pre-trip information search, they should also cover the whole travel process, providing full and integrated services for the travellers. The quality of these services can be evaluated and compared, if a framework is present, which contains all necessary aspects of journey planning and information provision. In order to provide high quality service in passenger transportation systems, the comparability of the information services has to be enabled.

Currently, plenty of different journey planners are available on the internet and many advanced information systems exist for transportation management and supervision. This paper focuses on journey planning in Europe; thus, only European journey planners have been considered. They provide several features, but in most cases, they are limited concerning the functions of multimodal and/or international journey planning.

Some academic papers have already handled this problem from a general perspective. Grotenhuis et al. [1] pointed out that travel information is an important factor, which highly contributes to the quality of public transport. Therefore, especially pre-trip information is essential for the passengers, and they consider travel time, alternative routes, and visualization as the most important parameters of journey planning. Keynon and Lyons [2] analyzed different levels of information provision and willingness to use alternatives. As a result, the more information was present, the more likely users chose better routes.

A suitable definition of multimodal journey planners is presented by Gentile and Nökel [3]. “These systems are front-end-back-end computer systems, which provide a traveler the best itinerary according to several parameters characterizing an intermodal passenger transport journey. Multimodal travel planners provide better modal integration and more sustainability by enabling travelers to select the most suitable combination of transport modes for the journey and could lead to an increase use of public transport, cycling or walking in urban environment.”

Multimodal journey planning has been a key part of different strategies to support the development of more sustainable transportation. A thorough study of the ITS (Intelligent Transportation Systems) action plan by Rapp et al. [4] aimed to support the development of multimodal journey planners by recommending functional, technical, and organizational specifications. One of the main conclusions of the EU-wide multimodal travel information roadmap [5] was that these services allow travelers to make informed choices by making them aware of all possible travel options.

Besides providing comprehensive information, journey planning should be also sustainable. Enoch [6] was dealing with the problem of planning travels of individuals and changing the policy framework to support sustainable transport. Lyons [7] claimed that the paradigms smart and sustainable need to be strongly aligned, and he proposed that a common framework for urban mobility development should be realized. Klecha and Gianni [8] investigated how the goal of sustainable urban mobility behavior can be reached by smart applications. They pointed out points that user requirements should be more taken into account, which would lead in the long term to a positive change in travel behavior.

Still, having journey planners with various features is not enough, and a comprehensive framework to evaluate multimodal journey planners is needed. The main research question is: “How to provide a quantified comparison of journey planners, which shows the strength and weaknesses of specific functions, so that travelers can evaluate and choose suitable journey planners considering user centric requirements?”.

Therefore, the aim of this paper is to develop a framework of aspects and evaluate several journey planners, taking into account the requirements important for the travelers. The paper is structured in 6 sections. Section 1 introduces the topic and gives an overview. Section 2 discusses contributions in the field of multimodal journey planning and evaluation of transport systems. Section 3 provides a framework of aspects, introduces user groups in order to enable the evaluation of multimodal journey planners, and describes the evaluation method, where the application of multi criteria analysis is presented. Section 4 reviews the selected journey planners and reports the results of the multi criteria analysis used on journey planners. Section 5 provides a comparison of the journey planners and used methods. The last section of the paper summarizes the main findings.

The main contributions of the paper are the following:

• A framework of aspects was developed for the evaluation of multimodal journey planners.
• User groups were introduced to include different requirements regarding journey planners.
• The functions of existing systems were evaluated from the user perspective.
• The evaluation provides quantified evaluation and ranking of multimodal journey planners, which supports choosing suitable journey planners for the users.

2. Literature Review

Multimodal journey planning has a broad literature. Several papers discuss concepts and frameworks of future systems with advanced functions. Some of them emphasize the importance of user requirements and personalized services, however only a few works discuss the functions of existing systems and provide an evaluation from the user perspective focusing on European journey planners.

In the field of multimodal journey planning, researchers created models and implemented journey planners, which at least partially realize advanced travel services. Zhang, Artenze, and Timmermans [9] described a framework of a personalized multimodal traveler information system, which includes real-time information, personal preferences, and multimodal route planning. Nuzzolo and Comi [10] handled in the i-TOUR project topics of adaptive, personalized multimodal routing with the combination of external information (e.g., road conditions, weather), location-based services, and personalized preferences. Spitadakis and Fostieri [11] presented WISETRIP application, which uses personalized information for multimodal journey planning, where passengers may set their preferences, receive alerts, and may reschedule their trip and choose alternative routes. In the work of Dib et al. [12], pre-trip information and several criteria (e.g., comfort and effort) were in focus in order to develop an efficient multiobjective routing system. Von Behren et al. [13] provided an integrated approach to define mobility parameters in an urban environment. A current application of Sierpinski et al. [14] is GTPlanner, which takes into account personal preferences when planning routes for the users and also provides information about their trips.

The objective of the paper by Guo et al. [15] is to describe the concept of an innovative multimodal journey planner. They claim that truly intermodal trip planners are currently not available and present the requirements for the development of trip planners. By studying the European trip planning framework architecture (FRAME), a conclusion is that the guideline provides necessary details on requirements, inputs, and outputs, however it is still difficult to see the overall picture from the user perspective. They also discuss challenges in the development and implementation of a truly multimodal trip planner (combining different transportation modes within a trip) considering mainly technical details and not that much requirement from the users.

The study of Wirtz and Jakobs [16] explored how users experience new technologies and how users perceive novel applications. As travel related applications are constantly evolving and changing, the users have to adapt to the new functions, such as planning a journey, buying tickets, receiving information about delays, and choosing from alternative routes. The authors have tested a multifunctional application and the results indicate that the application combines functions that support passengers through the whole journey. The impact of personal user attributes, such as age, gender, or membership in a certain technology generation is discussed. Providing well developed complex features for journey planning may be positively received by the users, however this has to be measured and evaluated.

After discussing personal preferences and recent developments of multimodal journey planners, the evaluation is in the focus of the second part of the literature review. The paper of Pérez et al. [17] reviews relevant literature on multi-criteria decision-making approaches for passenger transportation systems. From the analysis of over more than 30 years, the authors highlight the importance of taking into account the decision criteria of actors. Bulckaen et al. [18] combined multi criteria analysis with stakeholder analysis in order to evaluate and compared the ranking of different alternatives in mobility planning. More focused on travellers, the METPEX [19] project aimed to develop and evaluate a standardized tool to measure passenger experience and service quality across whole journeys.

The paper of Abenoza et al. [20] aims to investigate how travellers aggregate different door-to-door travel experience into an overall evaluation. They tried to find the parts of the trip which are most impactful. The results indicate that all trip legs need to be considered when evaluating the overall travel experience, especially trips legs involving waiting or transferring time. This means that all trip legs are found to be relevant in influencing the overall evaluation of the journey. However, not only the trip legs, but pre-trip and on-trip information has an added value to the personal travel experience of the users.

The study of Khoo [21] aims to investigate user requirements of app features, as travel information via apps may have a positive effect on changing travel behaviour. The authors used questionnaires to identify different characteristics of apps and routes of travellers. At the end, the traveller’s choice was analyzed. This paper also shows that well-functioning journey planners with suitable services are important for the users.

The paper of Lee et al. [22] aims to compare user interactions of dynamic travel applications in order to evaluate user experience. In the paper, three travel guide applications are tested to explore significant differences. The results indicate that the interface design has a real effect on user experience, thus, simple, straightforward visual displays generate satisfied users. This is truly an important aspect, which has to be taken into account during the evaluation of journey planners.

The paper of Ingvardson et al. [23] tries to understand the link between mode choice and travel satisfaction. A behavioural framework is presented, and a structural equation model is used to provide evidence that the interdependence exists, and personal travel experience is crucial when choosing transportation mode. A positive experience is not only related to the level of service parameters, but emotional and personal preferences are also influencing factors. This is also the case for the whole journey, where personal experience with journey planners has an effect on user satisfaction. Therefore, it is important to take into account the preferences of user groups when evaluating journey planners.

The study of Barbosa et al. [24] aims to understand how users evaluate available transport services using objective and subjective aspects. The authors claim that public transport services should play an important role providing high quality services for users. In order to analyze the perception of service quality, a multi-criteria evaluation method was created. Using this evaluation, it is possible to identify those objective and subjective factors that determine the opinion of the users regarding the service, which finally enables improvements.

Borkowski [25] stated that in Europe there are currently hundreds of journey planners, but truly multimodal journey planners are a rarity. In the paper, research on experience with developing travel planners has been conducted. Also, a framework for the introduction of journey planners is discussed and key barriers of real implementation are identified. A most interesting part related to the current paper is the assessment of selected European journey planners, leading to conclusions on features of an optimal multimodal journey planner. The listed features provided a solid basis for the elaboration of the proposed framework of aspects of multimodal journey planning evaluation. However, in this paper, only six very general aspects were analyzed, not focusing in detail on the user requirements. Furthermore, the aspects were only evaluated based on three categories and finally no quantified values were generated, thus no comparison and ranking between the journey planners was possible.

Although several papers deal with the evaluation of transportation systems and a few of them with the evaluation of journey planners, there is no application of multi criteria analysis on journey planners considering the user’s perspective. This paper aims to cover this gap and provide a method to evaluate and rank the journey planners.

3. Methodology

The most innovative aspect of the research is to use multi criteria analysis to evaluate journey planners with unique user-centric criteria and introduce user groups to refine the results. As a first step, the information services were identified, which are most important for the passengers. Then the user groups with specific requirements were formulated and the evaluation was performed.

3.1. Framework of Aspects

To the adequate functioning of a multimodal journey planner, a well manageable, user friendly interface and personalized adjustments (e.g., maximal number of transfers or preferred transport mode) are needed, which can provide some benefits for the users (e.g., time saving). In order to realize these prerequisites, reliable and actual data are needed. Also, the parameters have to be determined, which influence the suitability of the offered journey plans. These are beside the personal comfort of the social and environmental effects.

The most important aspects of multimodal journey planners were created considering existing definitions of the ITS action plan by Rapp et al. [4] (

Table 1. Framework of aspects.

1. Route Planning Services	2. Booking and Payment	3. Handled Data	4. Supplementary Information
ways of data input	tariff information	static data and personal information	comfort services
planning aspects	payment options	dynamic and estimated data	customer service
displayed data and visualization			environmental impact and equal opportunity

). The aspects (also called criteria) were collected in four main categories, containing single aspects. The detailed scoring of the single aspects is presented in the following listing in brackets. In general, for each single aspect, a maximum of 10 points could be given. The points for the aspects were provided by transport experts. Considering the certain features of the aspects, in most cases, where the maximum point was 2:

• 2 points were given, if the feature was well implemented or fully usable,
• 1 point was given, if the feature was partially implemented or partially usable,
• 0 points were given, if the feature was not implemented or was not unusable.

In the remaining case of features of aspects, where the maximum point was 1:

• 1 point was given, if the feature was well implemented or fully usable,
• 0 points were given, if the feature was not implemented or was not unusable.

Route planning services:

• Ways of data input (10 points): address (2) (e.g., roll down menu), name of stop (2), service facilities (2) (e.g., museums, restaurants, offices), GPS coordinates (2), pointing out on the map (2);
• Planning aspects (10 points): departure and arrival time (2), duration (1), costs (1), number of transfers (1), walking distance (1), and other aspects (4) (e.g., preferred transportation mode, P+R, B+R, crowding);
• Displayed data and visualization (10 points): compact design and easy understanding (2), visualization on the map (2) (e.g., zoom function and transport lines, transfer location plans), travel information (2) (e.g., travel duration with waiting times and distance), walking time and distance (2), alternative routes (2);

Booking and payment:

• Tariff information (10 points): prices (2), reduced fares (2), fee calculation of the planned route (2), way of data input for booking (2), possibility of choosing seats (2);
• Payment options (10 points): types of accepted bank cards (2), payment per mobile phones (2), location of ticket buying opportunities (2) (e.g., ticket automats), types of vouchers (4) (e.g., SMS, code per e-mail, paper ticket printed at home or at the station);

Handled data:

• Static data and personal information (10 points): timetables (2), creating a profile (2), setting personal preferences (2), saving searches and favorites (2), export features (2) (e.g., PDF, printing);
• Dynamic and estimated data (10 points): list of planned restrictions (2), visualization of planned restrictions (2), use of crowd sourcing data (2), information about actual traffic situations (2) (e.g., accidents, real-time travel information), providing alternative routes (2);

Supplementary information:

• Comfort services (10 points): Wi-Fi at the station (1), Wi-Fi on board (1), electrical supply (1), sightseeing (1), weather forecast (1), booking a room (1), car rental (1), opening times of shops (1), other services (2) (e.g., newsagent’s, bakery);
• Customer service (10 points): information in foreign languages (2), contact information via e-mail (2) and via telephone (2), feedback opportunities (2) (e.g., reporting a bug), forum (2);
• Environmental impact and equal opportunity (10 points): degree of air pollution and energy consumption (2), comparison of transport modes (2), routes for disabled passengers (2), information about vehicles (2) (e.g., low floor), webpage for visually impaired people (2).

3.2. User Groups

Since the appreciation of the information service depends significantly on the personal features of the travelers, different user groups were created from the travelers by their age (younger or older), mobility features (work or leisure based), and their mobility related abilities (without problem or handicapped). Travelers belonged to the younger group if they were under 30 years, while other travelers above 30 years old belonged to the older group. Travelers who commute to school or work at least four times per week are considered as work based travelers, while others are part of the group of leisure-based travels. Travelers who carry luggage or have motion problems (e.g., can travel only with a stick or wheelchair) belong to the group with burdens (handicapped), while other travelers carrying only hand luggage or having no specific motion problems are in the other group. The features of the user groups were defined by a short survey among transport experts, where the main features had to be ranked for each user group. The user groups were formed using the combination of these three points of view.

• Student (younger, work, and leisure based, without problem): Members of this group are open to new solutions, are interested in the use of dynamic data and new payment methods, but supplementary information is not so important for them.
• Worker (younger and older, work based, without problem): Daily travel in the city is their usual characteristics, that is, why they prefer the route planning options using actual data. Comfort services have the lowest priority for them.
• Tourist (younger and older, leisure based, handicapped): During leisure activities, the non-well-known routes are mostly used, therefore tourists (e.g., with big luggage) belong here. For them, route planning services and payment options are the most important, while dynamic data and supplementary information are the least important.
• Businessman (older, work, and leisure based, without problem and handicapped): Mostly, those passengers belong here, who want to travel in the most convenient way on time, and therefore they are more interested in supplementary information and dynamic data.
• Pensioner (older, leisure based, handicapped): Members of the elderly generation belong here, who can orientate and move mostly with difficulties. For them, the emphasis is on supplementary information.

3.3. Evaluation Method

The evaluation was performed in two main steps (Figure 1). First, the journey planners were compared (scoring) to each other based on the elaborated framework of aspects, which resulted in the general evaluation number. Then user groups, their characteristics, and their transportation share were defined. Taking preferences of the user groups into account (weighting), the average evaluation number for each journey planners was calculated. Finally, based on this new measure, the multimodal journey planners can be compared to each other. The elaborated method is based on the author’s previous works with his supervisor during the PhD studies [26].

For the scoring step, the multi criteria analysis (MCA) was adapted and implemented, because it produces clear and well-comparable results. MCA methods are easy to compute and widely used by Velasquez and Hester [27]. An interval-based scoring method was used, where the aim was to provide a comparison among the journey planners. The weighting (also using MCA) was introduced in order to take the different preferences of the user groups according to the aspects into account. The method has the following advantages:

• The methodology is easy and understandable, the process is simple.
• Simplification of complex situations and provision of a solution transparently.
• MCA is rational, which means that it provides a homogeneous result for a range of aspects, therefore the results can be easily compared.

3.3.1. Scoring

The multimodal journey planners (j) were evaluated by transport experts on a 0–10 valued scale according to the single aspects (i). To each route planner belongs I pieces of evaluation numbers. From the evaluation numbers and J pieces of single aspects. From the evaluation numbers, a I*J sized evaluation matrix was defined, in which elements are signed by p_ij. Summing up the evaluation numbers given for each single aspect, a general evaluation number can be provided for the multimodal journey planners (u_j).

$$u_j={\displaystyle \sum }_{i=1}^I{p}_{ij}$$

(1)

• i—aspects, i = 1, …, I,
• j—multimodal journey planners, j = 1, ..., J,
• p_ij—elements of the evaluation matrix,
• u_j—general evaluation number for the j. multimodal journey planner.

3.3.2. Weighting

The general evaluation number is already useful by itself, but it does not take into account the different preferences of certain user groups, which are defined later. The solution is presented by using normalization and weighting by transport experts.

All user groups (k) and aspects (i) have weighting numbers that belong to them, so called preference values (s_ki), which form a K*I sized weight matrix. The values of the elements in this matrix can be determined by exploring the preferences of the user groups.

From the evaluation matrix and the weight matrix, a K*J sized qualifier matrix can be generated, which takes into account the different preferences of the user groups. Its elements, which are the qualifier values (u_kj) for a certain multimodal journey planner and a certain user group, are to be calculated in the following way. The summed product is generated from the elements of the j. column of the evaluation matrix (p_ij) and from the elements of k. row of the weight matrix (s_ki), the value of which is then is divided by the summed product of the maximal given evaluation numbers (p_i^max) and the corresponding weights

$$u_{kj}=\frac{{{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{I}}{\mathrm{s}}_{\mathrm{ki}}{\ast \mathrm{p}}_{\mathrm{ij}}}{{{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{I}}{\mathrm{s}}_{\mathrm{ki}}{\ast \mathrm{p}}_{\mathrm{i}}{}^{\max }}$$

(2)

• k—user groups, k = 1, ..., K,
• s_ki—elements of the weight matrix,
• p_i^max—the maximal given evaluation number according to the i. aspect,
• u_kj—elements of the qualifier matrix.

Knowing the qualifier values (u_kj) for the multimodal journey planners and the transportation share (r_k) of the user groups, the average qualifier number (u_j*) can be determined, which is referred to all the passengers and takes into account the special expectations of the certain user groups at the same time.

$$u_j{}^{*}={\displaystyle \sum }_{k=1}^K{r}_k\ast u_{kj}$$

(3)

• r_k—transportation share of the k. user group,
• u_j*—average qualifier number for the j. multimodal journey planner.

4. Results

Such European web-based journey planners were selected, which are widespread or popular in a region, have leading ideas or extraordinary solutions. Europe is treated only as a testbed for the application proposed, while the framework is universal and could be applied to any journey planner in the world.

Firstly, their strengths and weaknesses were analyzed (

Table 2. Classification of journey planners.

		BKK Futár	TfL	AnachB	9292	Bayerninfo	Útvonalterv	Google Maps	Rome2Rio	TripGo	Eu-Spirit	RouteRank	DB	Scotty	Lufthansa
spatial coverage	urban	x	x	x	x	x	x	x	x	x	x		x
	regional			x	x	x	x	x	x	x	x		x	x
	international							x	x	x	x	x	x	x	x
handled transport modes	PT	x	x	x	x	x	x	x	x	x	x
	bike	x	x	x		x	x	x		x
	car			x	x	x	x	x	x	x	x	x	x
	rail		x	x	x	x	x	x	x	x	x	x	x	x
	air							x	x	x		x			x
	multimodal	x		x		x			x	x	x	x

). The selected systems were grouped by spatial coverage (urban, regional, international) and handled transport modes (public transport, car, rail, air, multimodal).

Considering urban and regional journey planners the BKK Futár from Budapest, the TfL (transport for London), the Austrian AnachB, the 9292 from the Netherlands, the German Bayerninfo, and the Hungarian Útvonalterv were selected, which provide basic journey planning with public transportation, but this service is usually extended to other transportation modes (e.g., bike, car, rail).

Some journey planners with international coverage were also considered, such as Google maps, Rome2Rio, TripGo, Eu-Spirit, and RouteRank. These services cover usually specific possible areas (e.g., Europe, Australia) in detail, but can plan between main destinations.

The basic railway or airplane-based journey planners, which are mainly service dependent, are the following: German railways (DB), Austrian railways (SCOTTY), and Lufthansa.

A few journey planners (BKK Futár, AnachB, Bayerninfo, Rome2Rio, TripGo, Eu-Spirit, RouteRank) provide multimodal services (combining different transportation modes within a trip), and in most cases they only combine a few modes (e.g., public transport with bike sharing).

4.1. Outstanding Functions

The innovative features of the journey planners were highlighted (

Table 3. Outstanding functions of the journey planners

	BKK Futár	TfL	AnachB	9292	Eu-Spirit	Bayerninfo	Útvonalterv	Google Maps	Rome2Rio	TripGo	RouteRank	DB	Scotty	Lufthansa
Route planning services	-	-	-	-	-	x	-	-	-	-	-	-	x	-
Booking and payment	-	-	-	-	-	-	-	-	x	-	-	x	-	-
Handled data	x	x	x	-	-	-	-	x	-	x	-	x	x	-
Supplementary information	x	-	-	x	x	-	x	-	x	x	x	x	x	x

). Progressive development has been realized by several journey planners, but the most outstanding ones are BKK Futár, DB, Rome2Rio, and Scotty. Concerning route planning services, many journey planners have already implemented a wide range of features. The developments mainly focused on handled data and supplementary information, while, concerning route planning services and booking and payment, no real novel solutions were presented.

4.2. Evaluation of the Journey Planners

Using the multi criteria analysis, the evaluation numbers and the weights were determined. Finally, journey planners were ranked by the average qualifier numbers.

4.2.1. Scoring

The evaluation numbers are presented and summarized in

Table 4. Evaluation of the journey planners.

	BKK Futár	TfL	AnachB	9292	Eu-Spirit	Bayerninfo	Útvonalterv	Google Maps	Rome2Rio	TripGo	RouteRank	DB	Scotty	Lufthansa
Route planning services	27	22	25	13	17	23	26	18	22	21	8	18	20	6
ways of data input	10	8	7	4	6	9	10	10	8	6	1	6	8	1
planning aspects	7	7	8	3	3	5	6	0	4	7	2	5	5	1
displayed data and visualization	10	7	10	6	8	9	10	8	10	8	5	7	7	4
Booking and paymen	6	9	6	9	0	0	1	4	1	6	3	16	9	12
tariff information	4	4	6	7	0	0	1	4	1	6	3	9	3	8
payment options	2	5	0	2	0	0	0	0	0	0	0	7	6	4
Handled dat	14	13	14	12	10	12	13	7	6	7	1	14	12	3
static data and personal information	4	6	7	8	2	6	5	7	6	5	1	8	6	3
dynamic and estimated data	10	7	7	4	8	6	8	0	0	2	0	6	6	0
Supplementary information	12	8	10	15	5	9	5	11	7	9	10	15	9	12
comfort services	0	0	0	0	0	0	0	5	0	1	1	4	2	5
customer service	8	6	6	9	5	5	5	6	5	6	5	6	6	6
environmental impact and equal opp.	4	2	4	6	0	4	0	0	2	2	4	5	1	1
General evaluation number	59	52	55	49	32	44	45	40	36	43	22	63	50	33

. The rows represent the aspects (i) and the columns represent the journey planners (j). The general evaluation numbers of the journey planners (u_j) are calculated by the summation of the evaluation matrix elements (p_ij) by columns. Not only the evaluation numbers, but the values in percentage are also shown. The maximal obtainable value was 100, where 10 points for each single aspect was the maximum. Values were assigned to the journey planners based on how much the journey planner met the requirements of the aspects.

In general, it can be obtained that urban and regional journey planners perform better regarding route planning services and dynamic data, while international journey planners tend to apply real multimodal solutions and provide balanced results for all main aspects except booking and payment. Finally, journey planners of rail and airway services providers have better results for booking and payment and supplementary information.

It has to also be stated that even the best journey planner did receive only 63% of all possible points, and on average the journey planners received 44%, which implies that there is still much development opportunity for the operators of journey planners.

According to the evaluation, the journey planner of DB provides the best services and widest range of information for passengers. This system is especially outstanding in the main aspects of booking and payment and supplementary information, but its dynamic information is also very detailed (e.g., environmental impacts and services at the station).

BKK Futár received second place because of its comprehensive features for route planning services and handled data. The system has implemented all possible data input modes (address, name of stop, POI (Points of Interest), GPS coordinates, and map) and many planning aspects and data display opportunities with detailed information and visualization. As BKK Futár is an urban journey planner, booking and payment is not that developed.

AnachB and Scotty were very close to each other running for third place. Both journey planners performed well regarding handled data, however AnachB has better results for route planning services, while Scotty is slightly better in booking and payment.

In the middle section are Google maps, TfL, Útvonalterv, 9292, and Eu-Spirit. All journey planners have outstanding results for some main aspects (e.g., Google maps for route planning services or 9292 for supplementary information), but their overall performance is not very spectacular.

Lufthansa, Bayerninfo, Rome2Rio, TripGo, and RouteRank received points below average, however they possess outstanding functions, which are not implemented in other journey planners (e.g., Rome2Rio and Lufthansa in supplementary information or Bayerninfo in handled data).

4.2.2. Weighting

As a first approach, estimated weights were assigned to the main aspects based on typical user group preferences and engineering considerations. In

Table 5. Weights of the main aspects.

	Route Planning Services	Booking and Payment	Handled Data	Supplementary Information	Transportation Share
Student	0.20	0.25	0.35	0.20	0.30
Worker	0.35	0.20	0.30	0.15	0.30
Tourist	0.30	0.35	0.15	0.20	0.15
Businessman	0.25	0.10	0.30	0.35	0.10
Pensioner	0.30	0.20	0.20	0.30	0.15
Average	0.28	0.22	0.26	0.24	-

, the weight matrix is illustrated, in the rows, the user groups (k), and in the columns, the main aspects (i). The weights of the single aspects (s_ki) are calculated by the equal distribution of the main aspect values.

The values of transportation share (r_k), which are shown in the last column of the table, are based on the results of the Hungarian national traffic data survey. This is a general value that may differ among countries; however, it still provides some guidelines.

The qualifier matrix (

Table 6. Evaluation of the journey planners considering user groups.

	BKK Futár	TfL	AnachB	9292	Eu-Spirit	Bayerninfo	Útvonalterv	Google Maps	Rome2Rio	TripGo	RouteRank	DB	Scotty	Lufthansa
Student	69	65	66	61	39	51	53	44	37	48	21	80	62	39
Worker	75	68	71	58	44	58	62	48	46	53	22	76	64	34
Tourist	68	64	64	58	34	46	49	45	40	52	25	80	61	45
Businessman	74	63	69	62	43	58	58	49	44	51	26	75	61	36
Pensioner	72	63	67	60	39	54	55	49	44	52	27	77	61	40
Average evaluation number	72	65	68	60	40	53	56	47	42	51	23	78	62	38

) can be calculated by weighting the journey planners according to the user group expectations and normalizing these values (u_kj). The values of the qualifier matrix were calculated considering how close they are to the maximal given value of single aspects. From these values, the average evaluation number (u_j*) can be obtained by multiplying them with the transportation share (r). On the basis of the formulas, the evaluation of multimodal journey planners was updated. In the rows of the table, the evaluation results considering user groups are present, while in the last row, the average evaluation number (u_j*) is shown, which refers to all passengers together considering their averages.

5. Discussion

Nowadays, several journey planners are available for users with various functionalities. The aim of this research was to provide an overview of European journey planners considering the perspective of the user and a qualitative evaluation of these journey planners based on specific aspects. Based on the results, it can be obtained that none of the journey planners have reached the maximum points, even the best journey planner has reached 2/3 of the maximum, which means that several features are not implemented in the journey planners yet.

Considering the score ranges of the main aspects, four categories can be defined.

• Fully elaborated: if a journey planner reaches at least 70% of the maximum score of a main aspect, it can be considered that the functions of the specific aspects are fully elaborated, thus it is recommended to the users,
• Partially elaborated: if a journey planner reaches at least 50% of the maximum score of a main aspect, it can be considered that the functions of the specific aspects are partially elaborated, which means that there are some useful functions,
• Moderately elaborated: if a journey planner reaches at least 30% of the maximum score of a main aspect, it can be considered that the functions of the specific aspects are moderately elaborated, thus only a few functions are available, which do not provide a perfect user experience and the usage is limited for the users,
• Not well elaborated: if a journey planner reaches less than 30% of the maximum score of a main aspect, it can be considered that the functions of the specific aspects are not well elaborated, thus the users cannot really use these features.

Based on the score ranges, it can be stated that in terms of route planning services, 50% of the journey planners are well elaborated, while almost half of the journey planners are not well elaborated considering handled data. There is no single journey planner, which would reach a well elaborated status in the case of all main aspects.

In order to obtain the differences between the general evaluation and weighted evaluation with user groups, the two evaluation numbers were compared to each other and the differences were highlighted.

The average evaluation numbers were by 9.4% higher compared to the general evaluation numbers of the journey planners (Figure 2). This relative significant modification is caused by the compensational feature of the applied MCA method. Because in the case of 7 out of 10 single aspects, the journey planners did not completely meet the features of the certain single aspect (namely the given values for any journey planner did not reach the maximal reachable 10 points). Therefore, the average evaluation numbers are consequently higher.

The other factor is the distribution of values given for the main aspects and requirements of the user groups. Namely, if all user groups prefer route planning services and in general the journey planners implement features of route planning services, the average evaluation number is higher. But in case of most user groups preferring supplementary information and the journey planners not having realized supplementary information, the average evaluation number is lower (even possibly less than the general evaluation number).

In our case, the distribution of user group preferences for main aspects was quite balanced and similar to the features implemented in the journey planners, and this factor did not change the final result of the average evaluation number. The biggest changes were obtained in case of DB, TfL, BKK, AnachB, and Scotty (between 12.7% and 15%), as they performed well regarding route planning services and handled data, which were the most preferred main aspects (with 0.28 and 0.26) of the user groups on average (see Table 5).

The elaborated method presented an evaluation and ranking of multimodal journey planners, which supports choosing the most suitable journey planner for the users. The method can be used to evaluate any journey planner considering the viewpoint of the travelers. Introducing the score ranges helps to identify the level of functionality, which is provided by the specific journey planners in terms of the main aspects. With the clear and quantified results, travelers will be able to review functionalities of the journey planners and make more conscious decisions when planning their journeys.

6. Conclusions

The evaluation and comparison of multimodal journey planners was performed before mainly in a descriptive way. The aim was to provide a quantitative evaluation method. Therefore, a framework of aspects was defined. Based on these aspects, an evaluation method was elaborated to compare and rank the journey planners. With the introduction of user groups, requirements of the users were also taken into account. The most important aspects from the passenger point of view are route planning services, booking and payment, handled data, and supplementary information.

Some European journey planners were selected, whose innovative features were highlighted. The developments mainly focus on the actuality of handled data and supplementary information, while concerning route planning services and booking and payment, no real novel solutions were available. The elaborated method helps the travelers to rate and compare information services from the viewpoint of the passengers. The introduction of user groups did slightly affect the evaluation numbers, and it highlighted some aspects (e.g., route planning, handled data), which are more important for certain user groups.

In general, it can be obtained that urban and regional journey planners perform better regarding route planning services and dynamic data. While international journey planners tend to apply real multimodal solutions and provide balanced results for all main aspects except from customer service information. Finally, journey planners of rail and airway services providers have better results for booking and payment and customer service information.