User Experience, Functionality and Aesthetics Evaluation in an Academic Multi-Site Web Ecosystem

Abstract

The present study focuses on using qualitative and quantitative data to evaluate the functionality, user experience (UX), and aesthetic approach offered by an academic multi-site Web ecosystem consisting of multiple interconnected websites. Large entities in various industry fields often have the need for an elaborate Web presence. In an effort to address the challenges posed by this need specifically in the field of academia, the authors developed, over a period of many years, a multi-site ecosystem within the Ionian University, which focuses on interconnectivity and a collaborative approach to academic content management. This system, known as “Publish@Ionio”, uses a singular content management infrastructure to allow for the creation of content for different websites that share both information and resources while at the same time allowing for individual variations in both functionality and aesthetics. The ecosystem was evaluated through quantitative data from its operation and qualitative feedback from a focus-group interview with experts, including website editors and administrative staff. The collected data were used to assess the strengths and weaknesses of the multi-site approach based on the actions and needs of the individuals in charge of generating content. The study led to conclusions on the advantages that interoperability offers in terms of digital and human resource management, the benefits of a unified aesthetic approach that allows for variability, and the necessity of collaborative content management tools that are tailored to the content’s nature.

1. Introduction

The generation and dissemination of content related to the research and educational activities of an academic institution on the World Wide Web is an involved process that requires a large amount of both digital and human resources [1]. In order to streamline this process, especially in larger organizations, which boast a multitude of different websites, each corresponding to various institutional offices or activities, this challenge should be tackled through the collaboration of all involved parties. But in order to achieve this, it is necessary to have access to the right digital tools.

To reap the technological and administrative benefits of interoperability and to accommodate the very specific needs of academic Web content management, the authors have been developing a multi-site Web ecosystem within the Ionian University, which has been used as the main component of the institution’s online presence for over 15 years. The goal of this research is to evaluate the operation of this ecosystem, known within the community of the institution as Publish@Ionio, through qualitative and quantitative means and to provide insight concerning the added value that such a system brings to the implementation of an organization’s content generation and dissemination strategy. The article focuses on the challenges of deploying and operating such a system in collaboration with specialized members of the community, as well as the identification of the various strengths and weaknesses of this approach.

The ecosystem’s evaluation process included both the collection of quantitative data based on real usage from the operation of the system and the collection of qualitative feedback provided through a detailed focus-group interview with expert specialized personnel, such as website curators, editors, and members of the various offices in charge of different institutional websites. The combination of qualitative and quantitative data leads to a robust body of information that was used to assess the strengths and weaknesses of the multi-site approach, as well as the habits and needs of people in charge of generating research and education-related content in the academic context.

This evaluation led to interesting conclusions on the advantages that website interoperability and collaborative content management offer in terms of maximizing the value of both digital and human resources and optimizing content manager UX, as well as the benefits of a unified aesthetics and functionality approach to visitor UX, and the necessity of custom content management tools that are tailored to the specific nature of content from the field of academic research and education. These findings do not apply solely to this specific ecosystem but are general indicators of the value of a multi-site approach that focuses on digital resource sharing and collaborative management features. Additionally, they provide insight into how this approach can become invaluable in addressing the need for an elaborate Web presence in multifaceted organizations.

In the following section, the research background of this research is briefly detailed. Afterward, a short description of the ecosystem is presented with emphasis on its main design tenets and its focus on collaborative content management. In the Section 4, the overall research design is detailed, and explicit research questions are posed, along with a description of the data collection process. The results of the data analysis are presented in the Section 5. Finally, the Section 6 addresses each research question individually, and a summary of findings, limitations, and future plans is presented in the Section 7.

2. Literature Review

2.1. The Evolution of Content Management

Over the past two decades, there has been a substantial evolution in the field of Web content management, with more and more vendors requiring advanced software solutions to achieve their content dissemination goals [2]. Web content management systems (WCMSs) are increasingly used by organizations to make Web applications faster, more flexible, and more cost-effective to develop [1,3]. Essentially, the rise and expansion of the Web accelerated the evolution of digital content systems, since they provide an easier way for Web content editors to create and publish content on a large scale [2].

Baker [2] defines content as “Information produced through editorial process and ultimately intended for human consumption via publication”. An early definition of content management by Boiko [4] describes it as the process of collection, organization, and publication of content. The systems themselves can be defined as software tools that simplify the processes involved in managing content efficiently [2] and store, organize, and share digital information [5]. A WCMS allows content providers to update a website without programming knowledge; it separates the content from aesthetics, and it allows for content and code reuse [6]. WCMSs are considered the best option for making website content changes easily in a quick and simple manner [7]. Additionally, WCMSs have emerged as pivotal instruments for facilitating the creation, administration, and maintenance of Academic digital content [8]. Selecting an appropriate WCMS requires more than just comprehending the content management requirements of an organization [9]. Various factors must be taken into account, including user interface, workflow management, content accessibility, and security [10,11].

Over the years, WCMSs have evolved from basic systems to smarter and more powerful tools that allow for greater customization in terms of aesthetics and functionality [11]. Advanced features may occasionally require the involvement of a developer, but the main purpose of WCMSs remains to serve as website management tools for non-technical workers. Hoffer et al. [12] asserted that only through the use of advanced WCMSs can organizations achieve an elaborate Web presence, featuring thousands of pages with frequently updated content. Bainco and Michelino [13] identified access control, content development, storage, transfer and publication, and workflow and time management as important features of advanced WCMSs used in publishing. Mican et al. [14] concluded that the key functionalities of an advanced WCMS must include source code manipulation, content organization through tagging, import capabilities from diverse formats, WYSIWYG (What You See Is What You Get) editor integration, restricted access controls, backup systems and more. Moreover, in terms of performance, Ravi et al. [15] demonstrated the importance of content caching techniques, and other studies [10,11] highlighted the necessity of security features and the identification of vulnerabilities. Finally, as Internet technologies move toward the future, WCMSs also evolve to encompass Web 3.0 technologies, content quality analysis features, AI algorithms, and more [16].

Another advanced feature of WCMSs is the ability to use a unified infrastructure to create multiple websites or a network of interconnected websites. This approach allows for both resource sharing and advanced management functionality, as well as advantages in deployment and upgrading [17]. Modern general-purpose WCMSs such as WordPress, Joomla, and Drupal all provide some sort of multi-site functionality [17,18,19] through either their core files or through plugins. On the other hand, this approach may cause problems with administrator confusion [17] or data integrity and security [18]. Rising up to the challenges posed by this multi-site approach in order to reap its benefits is a focal point of this research.

In the field of human–computer interaction, modern WCMSs are investing in simple and intuitive interfaces with a greater focus on personalization [16]. Specifically for task-oriented applications like WCMSs, the content manager UX design should support both novice and advanced users and aim to reduce workload and improve productivity [20]. One of the means that contribute to this is their visual aesthetics, which is considered a key factor of the perceived usability, credibility, and general quality of software systems’ user interfaces [21]. A high level of visual usability can influence both effectiveness and efficiency [22]. Lima and von Wangenheim [21] note that GUIs consisting of several visual components should work together in a pleasing way to help users accomplish their goals.

2.2. Academic Institutions as Content Providers

Within the spectrum of organizational classifications, academic institutions represent a distinct category, embodying roles as educational entities, hubs of scientific discourse, research institutions, and in some cases, corporate entities, among other designations [23]. Academic institutions, like other organizations, have the primary objective of communicating information [1]. Information regarding students, instructors, researchers, and administrative staff may be communicated at the university level in a manner that depends on the perspective of the stakeholders [24]. Academic information that concerns both the academic community and the general public is increasingly provided to users through the Web. This information includes research or educational activities, learning resources, and multiple other subjects that concern a wide variety of visitors. These visitors have different levels of technological aptitude, goals, and usage patterns, so academic websites need to be designed with everyone in mind [25].

Moreover, Internet content consumers seek not merely any content but rather information that is timely, relevant, legitimate, accurate, and comprehensive, and in response to this imperative, the use of WCMSs in an academic environment has become a necessity [1]. This use brings many challenges, including the incorporation of curricula [26] and the proper use of access management in both informational [27] and research-related content [28]. A subset of WCMSs with a focus on educational activities are learning management systems (LMSs) [29]. These are specialized authoring tools that include multiple systems for managing educational information [29,30] and boast a variety of interactive learning tools, ranging from static learning material to interactive multimedia edutainment experiences [31,32]. As demand grows, the availability and quality of such tools correspondingly expand to meet the rising requirements in this domain [33], with the focus remaining on course management, administration, and student–teacher network interaction, through an easy-to-use interface [34]. The online Web presence of academic institutions often works alongside LMSs and shares some of their management needs and characteristics.

Connell [35] asserts that one frequent theme in the field of academic content management is the difficulty of collaboration between academic libraries and the offices involved in website administration. This difficulty extends to other relationships within complex academic ecosystems. Yang et al.’s study in the field of interdisciplinary collaboration suggests that efforts toward better collaboration also serve departmental and disciplinary interests [36]. In a similar vein, the academic multi-site Web ecosystem that is evaluated in this study was developed as a means to address the challenge of collaboration in a manner that goes beyond simple workflow and access control. This ecosystem is concisely presented in the following section.

3. The Publish@Ionio Ecosystem

The multi-site Web ecosystem of the Ionian University, also known as Publish@Ionio within the institution’s community, is an innovative software, data, and file architecture, as well as a centralized information infrastructure and Web content management system, for the development and hosting of multiple interconnected websites. Although the ecosystem’s proposed architecture may be used for any group of websites, the focus of its implementation in the Ionian University is tailored to the needs of an academic institution. Since the ecosystem is primarily an architectural approach, it allows for flexible development and can be adjusted to fit any website development and hosting specifications. The ecosystem’s modular structure for the management (back end) and presentation (front end) of Web content also contributes toward a high degree of adaptability and a comprehensive coverage of Web presence requirements.

The basic architectural approach of the system is presented in Figure 1. The system uses a unique database and a singular content management platform to generate content for multiple interconnected websites. A common code framework exists both server-side, implementing the streamlined generation of the dynamic content of the various websites and their functionalities, and client-side, implementing a unified set of interactivity features and aesthetic templates.

The main goal of this structure is to create a central backbone of both data and systems that may be used by the various websites of the ecosystem without resource or organizational overhead. This reduces the requirements of individual websites while at the same time empowering the ecosystem as a whole. In a manner that mirrors the actual institutional structure, individual websites of offices may access pooled resources and disseminate information originally provided by individuals from different offices through a collaborative process.

On the front end, and in order to achieve aesthetic variety while ensuring optimized resource usage, the various websites utilize a series of common files, including CSS and JS files, as well as common HTML produced server-side by the frameworks’ libraries. Additionally, they make use of custom site-specific files that follow a common architecture but are heavily differentiated according to each individual website’s needs. Finally, the various content editors may provide additional CSS instructions from the content management platform and may also upload additional image files, not only as part of the content but also to use them in tandem with the custom CSS to enhance the website’s aesthetic presentation. An overview of this approach is presented in Figure 2.

The shared interactivity and aesthetic features that are the cornerstones of this approach aim to present visitors of the various websites of the ecosystem with a familiar experience that embodies the cultural values of the institution while at the same time allowing individual websites to be tailored to the needs of their audience as well as the specific nature of the content they publish. This approach achieves reduced resource requirements and allows for ecosystem-wide updates in functionality and aesthetics that propagate instantly to all websites. A major challenge when dealing with multiple websites is ensuring software tool versions are up to date, and the architecture of the ecosystem works in favor of achieving this objective.

As presented in Figure 1, the various websites also share a unique database and a singular content management platform, which uses a multilevel authentication system to not only ensure that different individuals have different access rights to the various websites but also that specific websites have access only to the entity modules and features that are useful for their particular content needs. The multi-level authentication system aims to facilitate management access to multiple individuals while ensuring proper compartmentalization of the responsibilities of each individual, thus avoiding internal conflicts.

The ecosystem’s data structure is based on modules that represent various informational entities inherent to the ecosystem. These entities, which are more than 30, are dictated by the needs of the academic environment as well as the nature of the content published by the websites involved in the ecosystem. Addressing the individual needs of a website through the development of a new module not only ensures that the purpose of this specific website is fulfilled but also provides a new tool to the ecosystem, which may be used to enhance the functionality of other websites with similar needs. The current modules and entities involved in the ecosystem are the result of over a decade of addressing individual needs.

For each entity item, the system keeps track of a wide variety of information, including the office that owns it, as well as the date it was created and last edited and from which content editor. The creation and editing dates and the content manager responsible for each item are important data, which are used to address conflicts. Moreover, they are utilized by this study’s statistical analysis, which is presented in the Section 5. Figure 3 presents an abstract overview of the database’s structure with the various entities and their relationships.

The overall architecture, which includes both the front-end and the back-end approaches, leads to the two major advantages of the ecosystem:

(1)

Optimized use of resources: Thanks to the use of shared software libraries, as well as the use of an appropriate file and data structure and system configuration, the ecosystem facilitates optimal digital resource usage. The system aims at maximum performance in terms of infrastructure overhead, management complexity, ease of debugging and upgrading, code execution speed, and other performance parameters. Moreover, the unified content manager experience aims to make it easier for content editors to achieve familiarity with the system, thus enhancing their performance.

(2)

High interoperability: Thanks to its modular architecture and its centralized database design, the system supports extremely high interoperability. The most prominent example of this is the ability to reuse or share content between the websites integrated into the system. In this scenario, the content creator retains control and management rights over the generated content, while at the same time, any website administrator retains control over their own website and may easily accept or reject any content-sharing requests. This process is enhanced further with an integrated notification system, thus implementing a complete collaborative environment.

This collaborative content management environment was designed in a manner that addresses not only technical challenges but also challenges related to the organizational duties and interpersonal relationships of the people in charge of content generation. To this end, the ability to accept or decline a share request, provide a comment on that decision, rescind the publication of an item, etc., were all implemented as integral parts of the process. The system notifying the involved individuals via email for actions related to sharing is also important, as it ensures transparency in the process and bestows agency to content managers. The ecosystem’s interoperability features, and especially the collaborative sharing process, are designed with special emphasis on respecting each content manager’s individual authority and preserving the delicate interpersonal balance between a multitude of specialized personnel of different rankings and responsibilities. Figure 4 presents an overview of the publication request process.

Beyond the ecosystem’s technical advantages, the specific implementation of the ecosystem in the Ionian University also provides features tailored to the generation of content related to research and education. These features revolve around the various data modules mentioned above, which include academic courses; research projects; institutional administrative processes; and academic activities such as seminars, workshops, etc., as seen in Figure 3. Content generation for these modules is closely related to the specific nature of academic activity and requires management tools that are tailored to its needs. The nature of these modules varies; some are more closely related to the institution’s content generation strategy, while some are more closely related to the actual institutional reality. The different nature of these entities affects their growth throughout the system’s evolution, as shown in the Section 5.

4. Methodology

In order to assess the strengths and weaknesses of the ecosystem presented in the previous section and to provide insight into the value that it provides, a series of research questions were posed. These questions guided the quantitative data collection process and provided the context through which the focus-group interview was designed. They are as follows:

RQ1:

In what ways can a multi-site Web ecosystem address the technical and administrative challenges of an elaborate academic Web presence?

RQ2:

How does a common development framework that allows for variations in aesthetics and features benefit the various websites of the different offices of an academic institution?

RQ3:

How do website interconnectivity and collaborative content management affect the generation and dissemination of academic Web content?

In this section, a brief overview of the research design will be presented, followed by details regarding the quantitative and qualitative data collection methods.

4.1. Research Design

The research design involved two different evaluation approaches that consisted of a focus-group interview, which is an inquiry evaluation method [37] that can be a very efficient way to collect user-driven UX and UI feedback [38], and a quantitative analysis based on content manager usage data, which is a type of user-centered evaluation method [37] that has the capacity to address usability and design matters [39]. Since the purpose of the evaluation was an assessment of the value of the overall ecosystem, the combination of two different methods was able to provide a comparative look, with the results of one method complementing the other [40].

Data-based approaches have been repeatedly used in research to study Web-related matters in terms of content, structure as well as Web usage [41]. In this case, the focus was put on usage-related information, since it could be conveyed through the data stored in the database. These metrics referred first and foremost to the use of the system by its content editors and administrators and had the ability to clearly present insights regarding content generation and interoperability. In a way that a log analysis or a Web analytics approach may lead to insights regarding the interaction between the end-user and the system [42,43], an analysis based on content-generation-related data may lead to insight regarding the interaction between the content manager and the system.

These quantitative data were further explored using feedback collected through a qualitative methodology, namely the focus-group interview. This traditional tool and its modern virtual counterpart [38] have the ability to help researchers gain a better understanding of the thought processes and patterns that motivate users to adopt a specific behavior [44]. It allows respondents to feel safer in the environment of their peers and thus leads to useful insights while also ensuring a shorter and less time-consuming process than individual interviews [45]. In this study, the focus-group method was used to receive feedback from experts responsible for various websites of the ecosystem. For these individuals, who are an inherent part of the Web content production process, the ecosystem is not just a service but an advanced software tool. In complex software engineering systems, in particular, focus-group interviews may be used in combination with quantitative data to acquire information from involved individuals such as developers or end-users [46].

This combination of database-derived quantitative data with qualitative feedback straight from specialized staff that are routinely interacting with the system creates a multifaceted evaluation process that has the ability to identify the causes behind negative and positive attitudes [47] and as such may also be used to provide useful insights concerning other similar advanced content management environments and multi-site general purpose WCMSs.

4.2. Quantitative Data Collection

The source of the quantitative data used in the evaluation of the ecosystem was the system’s database, as described in the previous section. The database consists of multiple data tables, corresponding to the various modular entities that comprise the ecosystem’s data structure. With full access to this database, which contains content from as early as 2006, the authors had the ability to collect information regarding the ecosystem’s evolution, the propagation of content, the utilization of collaborative content management features, and the effect of these features on the generation of both Web-oriented informational content and content closely related to the operational realities of the institution. The main means of collecting quantitative data on the system’s usage was through direct database queries to its various tables. Emphasis was given to information regarding the date and times and users involved in content generation, as well as the users’ engagement with the collaborative content management process. These metrics, which were a direct result of the content managers’ interaction with the ecosystem, offered insight into evaluating content managers’ UX.

The quantitative data analysis focused on exploring the growth of the ecosystem throughout the years by retrieving data from the users module and the offices module, which contained information about individual content editors and individual websites, respectively. Moreover, the analysis continued with data from the news module that included information about various news items relating to research and educational content. The news module was analyzed alongside the requests module, which contained data about the sharing requests for specific news items from specific users to specific offices. Finally, the analysis focused on monitoring the growth of different informational entities before and after the implementation of the collaborative mechanisms.

Table 1. Example of data from the requests module of the database.

#	Request ID	Office ID	Status 1: Accept 2: Reject	Request Datetime	Decision Datetime	Request User ID	Decision User ID
72	13095	4	1	28-11-19 06:40 p.m.	29-11-19 08:24 a.m.	1	14
73	13128	38	2	03-12-19 08:21 p.m.	03-12-19 08:30 p.m.	13	13
74	13130	56	2	04-12-19 01:13 p.m.	04 12-19 01:18 p.m.	13	22
75	13139	31	1	05-12-19 02:54 p.m.	05-12-19 03:02 p.m.	19	5
76	13136	41	1	06-12-19 05:33 p.m.	06-12-19 07:00 p.m.	1	15

presents a sample of five rows from the requests’ data table. Additional data are also publicly available (see Data Availability Statement at the end of this article).

4.3. Focus-Group Interview

The quantitative data collected through querying the database were complemented by qualitative feedback regarding the ecosystem, directly from expert individuals who interact with it on a daily basis. These individuals included people responsible for offices or activities that have their own specific websites; content editors from specific departments who implement their office’s Web content strategy; and members of the institution’s Webteam with wider editing capabilities, who also have a coordinating role during a website’s initial implementation. It should be noted that many participants were also involved with the university in an academic capacity, due to the intimate nature of the institution. As such, their feedback not only represented the opinions of content managers but also incorporated elements from the point of view of teachers, former students, and academic researchers. The members of the focus group were selected through purposive sampling in an effort to include people with a variety of roles. The total number of involved experts was nine, and the meeting was held on 3 January 2024. The meeting was coordinated by the research team and lasted a little over 90 min. Both the duration and the number of interviewed experts were within the customary values for focus-group interviews [44].

Table 2. A short summary of the focus-group interview participants.

#	Age	Gender	Use in Years	Attribute	Relation with the Ecosystem
P1	20–30	M	3	Graduate	Webteam designer and content manager for numerous websites
P2	30–40	F	6	Lab Researcher	Webteam content manager for all integrated websites
P3	20–30	F	2	PhD Candidate	Content manager for limited departmental and postgraduate program websites
P4	30–40	M	3	Academic Fellow	Content manager for limited departmental, postgraduate program, and activity websites
P5	50–60	F	11	Special Laboratory Staff	Webteam content manager for numerous institutional, departmental, and postgraduate program websites
P6	50–60	F	4	Associate Professor	Responsible for limited activity websites
P7	40–50	M	3	Assistant Professor	Responsible for limited departmental, postgraduate program, and activity websites
P8	40–50	F	2	Special Teaching Staff	Content manager for limited laboratory websites
P9	60–70	M	3	Professor	Responsible and content manager for limited institutional and departmental websites

presents a short summary of the participants with information about their relationship with the ecosystem and some limited demographic data.

The focus-group interview included an introduction section so that the participants who were not acquainted could become familiar with each other. After that, a series of guiding questions were posed in a non-strict manner. These questions revolved around three major axes, namely the aesthetics, functionality, and visitor UX of individual websites utilizing the ecosystem’s common framework, the content manager UX, and features offered by the ecosystem’s content management platform, and finally, the overall assessment of the system and the suggestion of changes or the addition of new functionality. The focus-group interview concluded with an open-ended evaluation question acting as a closing section, thus implementing all the major structure points customary in the focus-group methodology [45]. In terms of aesthetics, the qualitative evaluation focused on questions that aimed to determine whether the system’s capabilities were able to meet the website supervisor’s expectations and not a generic evaluation of the aesthetic value of the final product.

Table 3. A short summary of the focus-group interview questions.

#	Question	Group
Q1	Who are you and what is your relationship with the Publish@Ionio ecosystem?	Intro
Q2	With regard to aesthetics, to what degree were your initial expectations from integration with the Publish@Ionio ecosystem met?	Front-End
Q3	Which design features of your websites are shared with the framework and to what degree were the specific needs of your website met?	Front-End
Q4	To what extent did the functionality features of the ecosystem meet your website’s requirements?	Front-End
Q5	How satisfied are you with the user experience offered by the content management platform of the ecosystem in terms of usability and presentation?	Content Management Platform
Q6	To what extent are you satisfied with the capabilities offered by the content management platform in terms of content generation and website configuration?	Content Management Platform
Q7	Do you take advantage of the content management platform’s collaborative management features?	Content Management Platform
Q8	What are the strengths and weaknesses of the ecosystem and which needs are addressed by its use?	Evaluation
Q9	Which needs are not addressed by the system and what additions or changes would you suggest in terms of functionality and presentation?	Evaluation
Q10	What other remarks or feedback do you have to offer concerning the ecosystem?	Evaluation

presents an overview of the guiding questions.

5. Results

5.1. Quantitative Analysis

The Publish@Ionio multi-site ecosystem has been in production usage for the purposes of the Ionian University’s Web presence for over 15 years. In the past decade, the ecosystem has been expanded greatly, and at the time of this writing, it included 99 different integrated websites. They include the central website of the institution; websites for every department; websites for a multitude of institutional agencies and services; and websites for numerous postgraduate programs, laboratories, and events such as international conferences, summer schools, seminar courses, festivals, etc. From these 99 websites, 87 are bilingual, serving content in both the Greek and the English language, 11 have multiple versions (for example, yearly versions of a repeating conference), and another 11 support crowd-sourced content generation by allowing their staff members to edit content or submit news items for the consideration of their content managers. Figure 5 presents the expansion of the ecosystem throughout the years, by depicting the number of available websites every year since 2016. The continuous increase observed is a strong indicator of the system’s popularity within the institution’s community. Offices and teams that would use general-purpose WCMSs 10 years ago are now eager to become integrated into the ecosystem even for smaller endeavors that require an online presence.

In a similar fashion, an increase is observed in the actual number of content editors and administrators of the ecosystem. The total number of authenticated users at the time of this writing was 92. Out of these people, 2 had full administrative privileges, while the other 90 had access to specific websites. The number of specific websites varied, with an average value of 6.67 websites per user. The rate of increase in the number of content editors is presented in Figure 6. Although for most of its history, the ecosystem was managed by a smaller content team, in the past few years, with the expansion to more websites and the inclusion of people responsible for these new additions, the number of editors significantly increased, and this growth became the driving force behind the implementation of collaborative content editing features.

A major focal point of collaborative content management is the ability to share news items with other websites of the ecosystem. News items are an integral part of the institution’s content management strategy and involve announcements about research activities, events, calls, scheduling, etc. These items are often of interest to many people in the wider academic community and need to be disseminated through multiple websites. Figure 7 presents the number of unique news items for every year, as well as the number of total posts on the various websites for each such news item. The number of unique post numbers is 27,421, while the total number of posts reaches 40,080.

The number of unique news items displayed an upward trend as expected, but more importantly, a rapid increase was observed in the total number of posts after the implementation of the collaborative sharing request feature in 2019. This implementation, in combination with an increase in the total number of content editors, led to a much larger content footprint for the institution. This extra content is disseminated without additional overhead in digital or human resources and allows individual visitors who only engage with a limited part of the ecosystem to find information about a much wider range of activities.

The main representation of the sharing request feature in the ecosystem’s database is the “requests” module, which records every sharing request and information about it.

Table 4. A statistical overview of the requests module.

Subject	Total Distinct	Average Request per Subject
Unique news item	2126	3.40
Website receiving a request	57	126.98
Users making a request	42	172.33
Users deciding on a request	45	160.84

presents a statistical overview of the requests module, which includes the total distinct values and average values for each of the aforementioned information.

Out of 10,540 unique news items in the years after the introduction of the sharing request feature, 2126 were part of a sharing request (20.17%). Moreover, out of the 99 websites integrated into the ecosystem, 57 were involved in a sharing request (57.58%). Finally, in terms of the people involved, 42 out of 92 users made a sharing request (45.65%), while 45 out of 92 users accepted or rejected one (48.19%). These numbers show a very strong engagement of the content editor community with the collaborative request feature. On average, every news item that was involved in the sharing process requested to be reposted on 3.4 different websites. Figure 8 presents the total number of requests in the four full years the feature has been active. Each year, the number of requests increased dramatically as the feature gained even wider adoption.

It is also important to observe how content generation has evolved in other informational entities.

Table 5. New item entries per entity before and after the introduction of collaborative content management.

Subject	<2020	>=2020
Pages	881	2411
Courses	558	928
Staff	504	346

presents the number of new item entries in three entities before and after the introduction of the collaborative features. The content generation rate for informational pages increased dramatically, while at the same time, a small increase was noted for courses. On the other hand, staff entity items were not equally affected. This is an indication that the streamlining of the content generation and sharing process for news items has had a strong effect on the growth of informational entities that are more closely related to the institution’s online presence while having a limited effect on the content that is somewhat related to actual institutional activities and no effect on the content that is closely related to the institutional reality.

5.2. Qualitative Analysis

The qualitative feedback received via the focus group was transcribed using intelligent verbatim transcription and was analyzed on the basis of the questions presented in Table 3 of the Section 4. The consensus of the participants for each individual question is presented here, alongside some interesting remarks or differing opinions by specific participants.

5.2.1. Who Are You and What Is Your Relationship with the Publish@Ionio Ecosystem?

The introduction question served as a means for the participants to acquaint themselves with each other. Most participants briefly talked about their relationship with the ecosystem, especially the websites with which they were involved and their roles within the institution. Some briefly mentioned their first experience with the ecosystem with P1 and P6, noting the process of working with general-purpose WCMSs and P7 and focusing on the migration process of their website from such WCMSs to the ecosystem.

5.2.2. With Regard to Aesthetics, to What Degree Were Your Initial Expectations from Integration with the Publish@Ionio Ecosystem Met?

All the participants agreed that, through collaboration with the development team and due to the common styling base of the website layout, their initial expectations with regard to aesthetics were met to a very high degree. The general consensus was that the simple, clean, and user-friendly layout used in the websites is a major strength of the ecosystem and that the way information is presented in each website makes for a very efficient visitor UX. P7 described the departure from complex layouts found in modern websites as refreshing, and P2 was pleasantly surprised by the ease with which they were able to locate specific information and functionality; the neatness and order of the design evoked in them an emotion of pleasure. P5 noted that they were very pleased with the complete redesign of the website they manage, where no changes were made to the functionality and content of the website, while the aesthetics were fundamentally altered and conveyed the positive emotional reaction of their whole department.

5.2.3. Which Design Features of Your Websites Are Shared with the Framework and to What Degree Were the Specific Needs of Your Website Met?

The consensus among the participants was that uniformity and consistency between the websites of the ecosystem is essential. They also agreed that, despite the uniformity, the features of each website are distinct and represent their individual needs. P7 mentioned that the shared philosophy that is apparent among different websites makes them feel more comfortable. P3 found that the layout was very adaptable, although the basic structure was dictated by features that were commonly shared. P5 reported that visitors navigating between different websites are accustomed to specific layout features that these websites share and can always identify the sought-out information regardless of the websites’ individual aesthetics. P2 agreed that it is very important for the visitor to be able to quickly locate information on websites having distinct aesthetics within the same ecosystem, although P6 noted that, sometimes, uniformity is unattainable as most of the websites are introduced with particular requirements originating from the real-life characteristics of the offices or activities they represent online.

5.2.4. To What Extent Did the Functionality Features of the Ecosystem Meet Your Website’s Requirements?

All of the participants agreed that the functionality features of the ecosystem were more than enough to cover the needs of the websites they managed, and these features not only served their purpose with regard to their function but also greatly contributed to the overall aesthetics of the website. P7 noted that, through participating in managing many websites, they could not find a use case in which a functionality feature was not available to accommodate each website’s specific needs. As an example of specialized functionality, P3 mentioned the subsystem for submitting participation applications for educational programs, where applicants can submit information and upload documents through a simple and user-friendly online environment.

5.2.5. How Satisfied Are You with the User Experience Offered by the Content Management Platform of the Ecosystem in Terms of Usability and Presentation?

The general consensus among the participants was that the content management platform offered a very streamlined content manager UX and required very little expertise from the user. They agreed that through a basic training session, they were able to sufficiently manage the content posted on the websites. In particular, less technically adept users conveyed that interacting with the system evoked emotions of accomplishment and achievement when fulfilling a content management task and viewing the result. P1 stated that the platform manages to maintain a fine balance between the freedom given and the restrictions imposed on content managers in order to achieve optimal results, without overwhelming them with too many choices. They also mentioned remembering from their first interactions with the platform that its guidance feels like an embrace, especially to novice content managers, evoking an emotion of safety. P9 added that no previous experience in website management is required to use the platform, thus making it easy to use for non-specialized personnel through little training.

5.2.6. To What Extent Are You Satisfied with the Capabilities Offered by the Content Management Platform in Terms of Content Generation and Website Configuration?

The participants agreed that a powerful feature of the platform is the many automated operations that are provided throughout the whole process of creating and posting content, in order for the manager to be able to focus on the content itself. From creating content to posting on social media, the overall process was found to be streamlined and constantly guided by the system, either by suggestions or by warnings. P1 specifically emphasized how various features make content editing faster and enable the editors to be more productive. Moreover, the general consensus was that the most powerful feature of the platform is the manager’s ability to send and receive requests for posting content to and from other websites of the ecosystem.

5.2.7. Do You Take Advantage of the Content Management Platform’s Collaborative Management Features?

The consensus among the participants was that the platform’s collaborative management features are extremely efficient, organized, and very well structured. They all agreed that the alternative of having to contact each individual manager through various means in order to request the posting of a news item or an event is far from optimal. P7 argued that this collaborative system is innovative and unique among academic institutions and that it is very important for different departments within the same institution to be able to efficiently exchange information. P9 added that the overall process of making sharing requests is very efficient and time-saving for all parties involved and that the feeling of improved productivity makes the editors more keen to spend time generating content. P5 highlighted the ability to add comments when rejecting a request, thus providing the other party with feedback in order to efficiently filter their future requests. They also noted that the whole system is very democratic and emphasizes both the autonomy of each department and collectiveness.

5.2.8. What Are the Strengths and Weaknesses of the Ecosystem and Which Needs Are Addressed by Its Use?

All the participants agreed that the primary strength of the ecosystem is the uniformity between the websites providing the visitor with the sense that they are all part of the same family. This uniformity, combined with the simplicity in both aesthetics and functionality, makes for a very pleasing visitor UX. P3 specifically referred to the fact that a visitor will never land on a non-functional webpage, even when content is missing, and will always be given the ability to continue their browsing. In terms of weaknesses, the main complaint was centered around the fact that the content manager is not able to manipulate some of the website elements that are either fixed or inaccessible and thus will often need to work in collaboration with the development team. In contrast to that, P1 pointed out that the management system should be allowed to restrict the capabilities of the manager in order to sustain a balance between more and less advanced users on the basis of what absolutely needs to be editable by content editors and what may just over-complicate the content management experience.

5.2.9. Which Needs Are Not Addressed by the System and What Additions or Changes Would You Suggest in Terms of Functionality and Presentation?

The participants’ comments regarding needs that were not addressed by the system or additions to it were mostly related to content manipulation features and ideas about new technologies that could help improve the platform. A lot of the requests of the content managers had to do with limitations of their level of access. Additionally, some experts mentioned feeling constrained in various aspects and made requests that would allow them more agency. P6 proposed the integration of artificial intelligence (AI) features, for example, for translation purposes or for reviewing the content before posting, or even the use of generative artificial intelligence in order to introduce additional text to a post. Moreover, P6 also made suggestions for improved content dissemination through extra social media sharing features as well as integrated newsletter functionality.

5.2.10. What Other Remarks or Feedback Do You Have to Offer concerning the Ecosystem?

The final remarks of the participants were limited and varied on the basis of their relationship with the ecosystem. P6 proposed the creation of social media posting guidelines both in terms of how to generate content best suited for sharing and in terms of best practices for posting on the social media platforms themselves. Moreover, establishing a protocol for handling sharing requests within the ecosystem was proposed. P5 supported this endeavor, mentioning how request rejections might lead to misunderstandings when there are no explicit criteria for content dissemination within the ecosystem. They focused on the negative emotions evoked by a request rejection, and how they may be mitigated by clear guidelines regarding the sharing policy.

6. Discussion

In this section, the three main research questions introduced in the Section 4 are discussed through the prism of the results presented in the Section 5. For each research question, findings are detailed, and relevant references are discussed.

6.1. RQ1: In What Ways Can a Multi-Site Web Ecosystem Address the Technical and Administrative Challenges of an Elaborate Academic Web Presence?

As presented above, the multi-site Web ecosystem of the Ionian University uses a central data structure in tandem with a singular content management platform to organize and retain the content and data that are essential to the institution’s Web presence. These structural features, alongside its common code framework, are used to deliver dynamic content to a multitude of different websites. The ecosystem’s modular approach allows for versatility and ensures its ability to cover the specialized needs of academic content managers. Figure 5 and Figure 6 present the steady growth of the system’s reach within the Ionian University, especially in the past decade. The interoperability the platform offers through both its architecture and its focus on collaborative content management gains more and more value as the ecosystem grows, thus leading to a positive feedback loop. This interoperability establishes a high level of interconnectivity within the ecosystem’s websites, which has an established positive effect on the academic Web [48].

The process of migrating from an isolated website (usually operating through a general-purpose WCMS) to the ecosystem was mentioned as positive on multiple occasions by the expert content managers who took part in the focus-group interview. At the same time, the various participants shed light on a series of administrative challenges related to content generation and content sharing between institutional entities. The focus of the ecosystem’s singular content management platform on offering a streamlined content manager UX through ease of use and usability, even at the expense of optional sophisticated expert functionality, ensured that content managers of all skill levels could be productive. It should be noted that expert content editors who participated in the qualitative evaluation associated the content management platform’s use with emotions of accomplishment and safety, as well as a perception of improved productivity. This approach, which has a proven record in educational content management [49], makes the platform popular with a large number of content editors from various positions and, in combination with the system’s collaborative content management features, ensures the best use of their limited time.

Finally, part of the discussion and additions requested by the participants of the focus group were related to social media dissemination in an effort to attain better social media metrics (views, likes, etc.). This is indicative of the value of social media in content dissemination, which goes beyond just reaching new audiences but may even influence the visitors’ opinion in terms of the credibility or trustworthiness of the website and by extension the institution publishing the content [50]. For an academic institution, improving third-party perception through maximizing its social media metrics via a properly planned and executed Web content strategy is extremely important, and the ecosystem can be a valuable tool toward that end.

6.2. RQ2: How Does a Common Development Framework That Allows Aesthetics and Feature Variation Benefit the Various Websites of the Different Offices of an Academic Institution?

According to the qualitative feedback of the focus group, the combination of a common aesthetic and functionality core with the ability to produce multiple design variations tailored to the needs of each website was very popular with the participants. The existence of a unified approach was highly praised, and its results were multifaceted. Firstly, this unified aesthetic approach may be used as a means to convey the institution’s values. An academic institution encompasses a series of cultural characteristics that can be expressed through aesthetic features on the Web [51], and the ecosystem’s common design core allows for the permeation of these distinct parts of the institution’s character throughout the sum of its websites. From an ergonomics standpoint, common features and website structure mirror the attributes of locality and cultural similarity that are prevalent in the real-life relationship of offices belonging to the same institution. This backbone inherently influences the visitor UX of all the websites of the ecosystem, thus empowering the overall Web presence of the institution.

Moreover, the use of common features allows visitors to retain their familiarity with the layout when transitioning from one institutional website to another. This feeling of comfort and familiarity has been noted to lead to lower user disorientation and confusion, thus increasing both perceived and actual usability [52]. It was explicitly noted by the experts that the common positioning of functional elements or familiar interactivity had a positive effect on their engagement. Familiarity is an achievement of the human factors approach of the system’s design, which transfers the interrelationship of the various offices of the institution from reality to the Web [52,53] and as such has a positive effect on usability and the overall visitor UX [54].

Moving beyond the common base of the ecosystem’s features, individuality and adaptability were also highly valued by the participants. The variety of scientific fields that are covered by the institution’s departments was large enough to warrant serious differentiation both in terms of aesthetics and functionality. Participants were vocal about their ability to influence the aesthetic result of their websites. This is something to be expected since aesthetic elements are closely related not only to the appeal of a website [55] but also to the values it is trying to convey [49]. As mentioned in the focus-group interview, aesthetic variation is also the result of the requirements posed by the pre-existing real-life entities that dictate the creation of a website.

6.3. RQ3: How Do Website Interconnectivity and Collaborative Content Management Affect the Generation and Dissemination of Academic Web Content?

Interconnectivity is one of the major strengths of the Publish@Ionio ecosystem since it not only facilitates the dissemination of informational content far and wide throughout the ecosystem but also allows changes in this content to be propagated instantly. One of the most important factors of website evaluation is ensuring the relevance, accuracy, and timeliness of information [56], and the ecosystem’s content organization and dissemination practices play a great role in achieving these goals.

The collaborative features received a lot of praise from the interviewed experts, especially with regard to saving time and ease of use. This pivotal importance of the “requests” feature is signified even more by the chart presented in Figure 7. With the introduction of this feature, the increased footprint of news item reposts on different websites displayed radical growth. This increased content presence may lead to SEO benefits, which are closely related to the content’s quantity and textual richness [57].

Table 4 and Figure 8 present the high percentage of involvement of both individuals and websites with the collaborative request system, as well as the system’s continuous rapid growth, as measured quantitatively through the requests’ data entity. Through both qualitative and quantitative means, the collaborative process was deemed to be useful and efficient, but ultimately, it also serves as a motivation for a website to go through the process of migrating and integrating with the multi-site Web ecosystem.

In terms of content generation that is not related to news dissemination, Table 5 provides some interesting insights. Entities that are closely related to Web content generation such as informational pages have seen a rapid increase in content quantity in recent years, while entities like courses, which serve a more practical purpose, show a rate of growth parallel to the general growth of the ecosystem as it integrates more offices of the institution. On the other hand, the growth of entities such as teaching and administrative staff, which are very closely related to the logistical realities of the institution itself, remain unaffected by the Web content growth. This is expected since Web content strategy does not have the ability to easily influence other factors of institutional reality, but considering that a documented correlation between the Web presence of an institution and its academic excellence exists [58], it is not unreasonable to expect that overcoming the challenges of academic content management and ensuring a large and high-quality Web presence may eventually influence the institution itself, especially in the modern world of global interconnectivity.

7. Conclusions

This study focused on presenting and evaluating the academic multi-site Web ecosystem of the Ionian University, known within its academic community as Publish@Ionio, and the solutions this ecosystem implements in terms of aesthetics, functionality, content manager UX, as well as website visitor UX. The evaluation consisted of quantitative data directly derived from the ecosystem’s data structures and referring to real-world usage of the system, as well as qualitative feedback collected through a focus-group interview with expert individuals who interact with the system on a daily basis through their roles within the academic community of the institution.

The ecosystem boasts an architectural approach that, according to the evaluation presented in this research, ensures the optimization of the performance of digital and human resources, which, in combination with its focus on providing a content management UX that puts emphasis on ease of use and usability, constitutes a useful tool for implementing an institutional Web content management strategy and enhancing and enlarging the online presence of an academic institution. Moreover, this approach ensures a unified aesthetic identity that allows the institution to promote its culture while offering the added benefit of evoking feelings of familiarity to the visitors of different websites within the ecosystem. This common framework does not hinder the system’s ability to also offer unique aesthetics and tailored functionality among its variety of different websites. Finally, one of the ecosystem’s major driving forces is its focus on interoperability and a collaborative approach to content generation and dissemination, which ensures a stable growth of the institution’s Web presence by making the best use of its resources.

In the future, this research may involve an even deeper exploration of the quantitative data detailing the ecosystem’s function through log analysis or the use of custom-built analytics to measure both general public traffic and how it flows between the various websites of the ecosystem. Additionally, the aesthetic evaluation can be expanded to not only investigate the system’s capabilities of meeting manager expectations but also include quantitative and qualitative data that objectively measure the pleasure derived from the final aesthetic result, as well as the emotions evoked by specific aesthetic choices. In the same vain, a more in-depth visitor UX study focusing on various websites of the institution and including more advanced evaluation techniques, with added emphasis on ergonomics and sentiment analysis, may lead to an improved common framework that will not only allow for the optimization of the sum of the websites integrated with the ecosystem but will also provide the research community with more insight into improving the Web UX of various types of academic websites. Moreover, the implementation of AI features, including content analysis, translation services, summary generation, and image and multimedia manipulation, is underway, and future research can focus on the integration of these features and their adoption by the ecosystem’s content managers.

Although this study focused on the existing ecosystem, which might be influenced by other factors beyond the scope of this research, its findings may serve as guidelines for using modern Web technologies to combine the fractured Web presence of any academic institution into a whole that is greater than the sum of its parts. Beyond that, the overall architectural approach of the ecosystem presented in this study may not only be useful for academic institutions but also for other private or public organizations that wish to streamline their large multifaceted online presence.