Digital Twin and Cloud BIM-XR Platform Development: From Scan-to-BIM-to-DT Process to a 4D Multi-User Live App to Improve Building Comfort, Efficiency and Costs

Abstract

Digital twins (DTs) and building information modelling (BIM) are proving to be valuable tools for managing the entire life cycle of a building (LCB), from the early design stages to management and maintenance over time. On the other hand, BIM platforms cannot manage the geometric complexities of existing buildings and the large amount of information that sensors can collect. For this reason, this research proposes a scan-to-BIM process capable of managing high levels of detail (LODs) and information (LOIs) during the design, construction site management, and construction phases. Specific grades of generation (GOGs) were applied to create as-found, as-designed, and as-built models that interact with and support the rehabilitation project of a multi-level residential building. Furthermore, thanks to the sharing of specific APIs (Revit and Autodesk Forge APIs), it was possible to switch from static representations to novel levels of interoperability and interactivity for the user and more advanced forms of building management such as a DT, a BIM cloud, and an extended reality (XR) web platform. Finally, the development of a live app shows how different types of users (professionals and non-expert) can interact with the DT, in order to know the characteristics with which the environments have been designed, as well as the environmental parameters, increasing their degree of control, from the point of view of improving comfort, use, costs, behaviour, and good practices. Finally, the overall approach was verified through a real case study where the BIM-XR platform was built for energy improvements to existing buildings and façade renovations.

1. Introduction

The Lombardy region hosts more than 60% of the existing Italian building stock, most of which was built before the 1970s, resulting in low performance in terms of overall final energy absorption and corresponding CO₂ emissions, with an average primary energy demand for heating of about 202 kWh/m²y [1]. With appropriate support, buildings could lead the transformation of the EU energy production and delivery system. An energy renovation of the building stock, in fact, could reduce the energy demand by 80% by 2050, compared to 2005 levels [2].

The energy renovation market is emerging in Europe, and since the financial crisis period, it has started playing a major role as a stabiliser of the building sector and, consequently, of the overall European economy. In line with this, the construction sector has also been pointed out in the Italian context as a key factor for achieving the long-term objectives set by the country [3].

The Italian “Action Plan for Energy Efficiency” (PAEE, 2019), among others, established: (i) the strengthening of minimum energy performance requirements for new buildings and the refurbishment of existing ones, progressively leading to an increase in nearly zero-energy buildings (nZEBs), in line with the EPBD Directive, and (ii) the consolidation of the tax deduction system for the energy refurbishment of existing buildings. Several research works have contributed to this topic, presenting the impact of different energy conservation measures on the building stock to reduce greenhouse gas emissions. Promoting innovative renovation measures for such buildings is indeed crucial for reducing the carbon footprint and supporting the decarbonisation of the building sector.

The combination of pioneering methods, latest-generation digital tools, and software applications can be crucial in both supporting building management needs and fostering the transmission of computer-generated perceptual information for both expert and non-expert users. To that end, the present work proposes a novel holistic approach to energy-efficient envelope retrofitting of multi-storey and multi-owner buildings, through the development of a building information modelling (BIM)-based digital twin (DT) and a cloud extended reality (XR) platform. The proposed retrofitting strategy foresees modular prefabricated components that need no additional finishing on site and no scaffolding for their installation, and that promote novel design approaches (Figure 1).

The present paper illustrates the developed methodology and the actions carried out on a case study building in Cinsello Balsamo (Italy) that was used as a demo site for two research projects. The case study building was firstly considered as an Italian pilot action of the EASEE project (Envelope approach to improve Sustainability and Energy Efficiency in existing multi-storey multi-owner residential buildings [4]). In this first stage of the research, activities were focused on optimising the scan-to-BIM process for energy refurbishment. In the second stage, activities on the building continued within the HOMeBIM liveAPP project (Development of a Multi-user live APP of 4D home virtual reality for the improvement of comfort-efficiency-costs, from a cloud platform that controls the BIM-sensor flow over time [5]). In the second step of the research, the main aim was to integrate the results of the “scan-to-BIM” process and sensor data into a simple user interface. As further described in this paper, developing a digital twin and a cloud platform allows the authors to support different types of users and improve the real-time management of the building’s energy comfort.

The remainder of the paper is structured as follows: Section 2 highlights the motivation and the main contributions of this paper; Section 3 presents a review of related works and outlines the current gaps between BIM and DTs; Section 4 reports the activities carried out on the research case study focusing on data integration presenting the developed solution for bridging BIM data into a DT; Section 5 presents some results in terms of comfort analysis and cost reduction obtained as a consequence of the research activity.

2. Motivation and Background

Between 1925 and 1975, there was little or no consciousness about the energy-efficient design of buildings, and such a building stock is therefore now responsible for a considerable amount of energy demand. More than half of the European buildings belong to such a category, ten million of them being multi-storey buildings with distributed ownership. Taking into account renovation processes, external envelope retrofitting has a significant impact on the occupants’ quality of life and on the overall refurbishment working days. In particular, traditional retrofitting approaches need scaffolding on the outer façade, requiring occupants to seal the windows, and introducing safety issues. Furthermore, besides being an additional cost (e.g., local taxes to occupy the public property), scaffolding can also negatively affect both traffic and passers-by. Hence, considering that, for a seven-storey building, this process can take, on average, between 12 and 24 months, the need for a new approach is quite clear.

With particular regard to reducing refurbishment operation times, such an approach requires simplified fault-tolerant procedures and ease of application to achieve the expected building energy performance successfully. The retrofitting method proposed by the EASEE project is based on high levels of prefabrication to limit additional on-site operations. Nevertheless, a novel approach must be defined to properly manage the retrofitting intervention, from accurately measuring the building envelope and proper design optimisation to the final installation of the panels.

The EASEE project also offered several other strategies for the outer façade of buildings towards energy-efficient envelope retrofitting, as will be further elaborated in this paper. In particular, the influence of the adopted solution on the definition of the developed approach is also discussed, paying specific attention to other non-technical parameters such as the cost of the intervention and optimisation of the installation time.

As previously mentioned, the retrofitting concept is based on minimising on-site operations to reduce the construction yard timing by using prefabricated components that can be easily installed. The choice of a retrofitting method based on a high level of prefabrication with components increases the need for accurately measuring information of the building envelope to manage the design and installation. For this reason, the characteristics of the solution adopted for the outer retrofit directly influenced the design workflow. In particular, the main design challenges that had to be faced were:

• The availability of as-found geometry information according to production and installation tolerances;
• The definition of an optimisation strategy to determine the panel configuration;
• Identification of the best anchoring size and length;
• Materialisation of anchoring positions to support the panels during installation.

In order to efficiently face the previously listed issues and manage the design procedure efficiently, a specific workflow was developed.

The applied solution for the outer façade significantly impacted the strategy adopted for the definition of the “as-found model” to be used for the design of retrofitting in terms of production, installation tolerances, and installation procedure. In particular, a simple manual survey of the façade was inadequate since the achievable accuracy (in the order of some centimetres) was insufficient to identify the best anchoring size and length (a few millimetres). Accordingly, a specific survey strategy was necessary to achieve the requested accuracy of the “as-found” building model based on point cloud data of the façade acquired with terrestrial laser scanning (TLS). The first step of the developed strategy was materialising a stable local reference system (LRS), allowing simple repositioning during different epochs of the survey and serving as a unique reference for information localisation and design.

Once the LSR was established, scans discretised building façades with 1–2-millimetre accuracy. The discretised model of the building can be used to evaluate the existence of façade irregularities (i.e., a non-planar façade or significant out of plumbs) whose existence directly influences the choice of the best anchoring size and length. A simple CAD-based approach for panel size and position design was inadequate to optimise the panel configuration (i.e., minimising the number of panels with different sizes). For this reason, scan-to-BIM-based modelling was exploited. The main advantage of this approach is a specific parametric “panel” object. In particular, the “panel” can be designed such that its height (“z”) and length (“x”) can be easily set by changing a specific parameter while stratigraphic information is kept as primary information. This feature of the BIM object fits well with the modularity of the panel, since changes in its size can be easily made without the need to revise the overall project. A further advantage of a BIM-based approach is its intrinsic 3D feature. Indeed, once panels are designed, their 3D position is also defined in the LSR. Those positions can be transferred to a total station used to trace and materialise the localisation of the anchoring during the first stages of the construction yard.

The second step in the presented research was carried out during the HOMeBIM project. In particular, the HOMeBIM project focuses on developing “intelligent living”. At present, the process of generating and managing a BIM model presents, among others, the following bottlenecks: (i) the generation process is mostly manual, especially in the case of “as-built” models; (ii) the management of large amounts of relevant data in real time is not directly managed by BIM modelling software, complicating data analysis and feedback actions; (iii) interoperability between BIM models and energy models is an aspect not yet fully guaranteed.

Although BIMs are potentially information-rich tools that can be integrated and controlled over time, they are still underutilised systems. The EuPPD Directive (Public Procurement Directive) offers for their adoption in all EU countries rewarding or mandatory measures as a vehicle for savings (from 5–15% to 20% during the entire asset management cycle over time): their integration and interaction with the live app aims to exploit their wealth and make the app usable in a simple way by diversified users, making the “home” the philosophy of BIM control and combining it with interactive and easy-to-use intelligent systems typical of VR. For this purpose, an easy-to-navigate live app was implemented to support the use of data: the inclusion and circular involvement of citizens in the technical energy supply chain are an integral part of the EU zero energy objectives to be achieved by 2050. HOMeBIM has therefore created a simple, low-cost tool to help the inhabitants of a building control the comfort level of their housing units and improve it through the adoption of a simple app interacting with easily installed low-cost internal sensors for the measurement of environmental and external parameters for meteorological monitoring.

The key outputs of this process can be summarised as follows:

• The development of a cloud platform to host and share scan-to-BIM projects, managing the respective big data, such as point clouds from laser scanning and digital photogrammetry (primary data sources), as well as reports, digital drawings, and multimedia (secondary data sources);
• Improved efficiency in terms of workflow, coordination, and collaboration, with user-friendly 3D visualisation;
• Improved accessibility of VR projects (i.e., sharing executable files to install dedicated apps);
• AR object implementation and sharing;
• Improved interoperability of digital models, through specific proprietary and open-source exchange formats;
• Diversification of the digital uses (e.g., smart glasses, VR headsets, PCs, mobile phones, tablets).

Further IT development aims to improve the interactivity of the platform, especially by connecting the monitoring data. The final goal is, in fact, to integrate digital models and real-time data into a single digital solution, to support awareness and building management over time.

3. State of the Art

The digital revolution in the AEC sector has allowed professionals to improve their analyses, projects, and building management [6,7]. In the last decade, interesting results have been achieved in energy monitoring and the comfort of buildings [8,9,10,11,12,13]. Thanks to the introduction of control and simulation tools such as building information modelling (BIM), it has been possible to move from a 2D vector CAD representation to a digital building representation capable of communicating different levels of information (LOIs) [8,9,10]. As is well known, in this field of applied research, 3D digital information representation has communicated and shared information automatically in both textual and numerical terms. In particular, BIM has made it possible to incorporate physical and mechanical information of the materials used into parametric objects, create and share the masonry stratigraphies, and represent the constructive logic following specific international standards associated with them aimed at different levels of study such as the level of development (LOD) [14]. Once the materials have been identified and characterised in physical and mechanical terms, it is possible to ask the BIM software to automatically extract schedules and databases capable of interacting with external data sources [15,16,17,18]. In this way, descriptive fields and specific BIM parameters (default and custom) can be associated with quantities.

Consequently, the computational aspect is decisive for most disciplinary sectors for preparing calculations and simulations addressing the management of the building during its long life cycle (LLCB), both for existing buildings and new constructions [19,20,21]. Volumes, areas, and costs automatically extracted from BIM projects have become fundamentals in calculating structural simulations, energy analyses, and conservation and preservation projects [22,23,24,25,26,27,28,29,30]. Consequently, the concepts of geometry closely related to the value of measurement, accuracy, and reliability have become a key research field in recent years [31,32,33,34,35]. These studies have clarified the need to go beyond the LODs and LOIs proposed by the main international standards, trying to clarify the need to create and model based on unique parametric objects both from a morphological and typological point of view.

In this context, 3D surveys and the latest-generation developments in the scan-to-BIM field have made it possible to have an increasingly reliable and valuable basis for the existing building generation process [36,37,38,39,40].

The process has been investigated and defined by many studies, which have been able to arrive at very reliable values in terms of accuracy and precision, passing from simple point clouds originating from laser scanning and digital photogrammetry to informative models able to represent the complexity of existing buildings, archaeological sites, and complex infrastructures [41,42,43].

The urgency to share information associated with BIM projects has led research and development to investigate advanced forms of interactive sharing, trying to unite all professionals and those not involved in a single digital process. For this reason, studies and software manufacturers, after an initial phase of updates of their most used interfaces such as Autodesk AutoCAD, McNeel Rhinoceros, Graphisoft ArchiCAD, and Autodesk Revit for the management of point clouds, have moved on to the development of BIM cloud environments [44,45,46,47,48]. Cloud sharing related to BIM projects has brought several advantages, including (i) collaborating on native data in real time [49,50,51], (ii) providing immense storage space cheaply [52], (iii) viewing digital models and information interactively via the web [53,54], and (iv) facilitating the storage and accessibility of different types of 2D and 3D formats [55,56,57]. Despite their many benefits, BIM clouds also have disadvantages, including technical issues related to security, connection, and attention [58,59,60].

On the other hand, information flow is purely static even if shared online through the latest software interfaces. For this reason, the transition from BIM to DT has been crucial for the real-time monitoring of buildings in recent years [61,62,63,64]. The emergence of Internet of Things (IoT) applications has offered numerous new digital solutions [65,66,67]. In particular, the ability to connect in real time to online sensors distributed in an environment has led to the DT concept of the built environment, offering a high-level representation of buildings by integrating the physical and digital aspects, improving simulations and resource design, from project execution and resource operation to communicating information and data throughout the life of the building [68,69,70].

The term “digital twin” was first proposed in 2003 but has only gained more popularity recently. One of the first real DTs can be considered the project developed by the National Aeronautics and Space Administration (NASA). The DT was created to continuously assess a spacecraft’s state to mitigate the deterioration of the vehicle [71]. Later, DTs were implemented and used in the construction field to reach a mirror and digital representation of the actual production process, integrating physical processes with virtual products and relevant connection data.

As a result, unlike BIM clouds, real-world synchronisation with a single digital hub for control of the construction process, environment monitoring, facility management, and other life cycle processes in the built environment has become a research sector to be further explored.

Recent studies have highlighted advances in enabling technologies and establishing a convergent context in a single web platform solution, conducting a systematic review to identify the development of emerging technologies that facilitate the evolution of BIM to DTs in built environment applications [72,73]. However, developments in the built environment are still in their embryonic stages. A careful analysis of the main results reveals the difficulty of achieving a high level of development based on BIM, DTs, and web platforms capable of sharing monitoring data in real time. The primary research challenge is pursuing a deep cyber-physical integration between digital models and guaranteeing a broad applicative perspective in representing, predicting, and managing the infrastructure’s current and future conditions. Leading companies such as Siemens, British Petroleum (BP), and General Electric (GE) have developed and patented platforms that use DTs for technical product innovation. In the construction field, some researchers have guaranteed a complete solution for monitoring the construction process in real time, adopting advanced augmented/virtual reality (AR/VR) equipment coupled with sensors to acquire images or videos of the physical site, useful in safety monitoring, risk warning, and remote education [74,75,76].

Furthermore, it should be recognised that the fusion of the scan-to-BIM process with heritage models (HBIM) is allowing users to improve the automation of the geometric modelling of complex scenarios, achieving spatial precision and high metric control [77,78]. Other interesting studies have improved decision making and predictions among stakeholders by reconstructing and visualising the acquired state of the built environment through the use of 3D point clouds, IFC formats, damage data analysis, and repair histories to guide long-term strategies for the evaluation and maintenance of infrastructures and urban centres [79,80,81,82]. However, with emphasis placed mainly on 3D geometry and model evaluation in digital twins, less attention has been paid to developing XR platforms capable of sharing this large amount of data in real time.

For these reasons, this research proposes a process capable of optimising DTs through automatic data collection, conceptual development, and dynamic analysis in diagnosing problems to optimise design, operation, control, and maintenance. In other words, through the automatic detection of specific parameters and a specific IT implementation, this study proposes the development of a mobile app and an XR web platform for the real-time evaluation of the thermal conditions of a building, favouring the formulation of optimised solutions based on the data and put into operation in good time to bring benefits in terms of superior reliability and efficiency. Therefore, the development process applied to create a digital toolkit capable of providing a rich and constantly updated flow of data on the energy and performance characteristics of BIM and IoT sensor networks is outlined both for latest-generation devices (laptops, workstations, VR headsets) and for different types of users (professionals and tenants of the building object of this research).

4. Research Case Study

4.1. The Research Case Study: From 3D Survey to as-Found Building Information Modelling (BIM)

The EASEE Italian case study is located in Cinisello Balsamo (Via del Carroccio 18), in the metropolitan area of Milan. The high-density settlement presents a concentric pattern, predominant in the public housing estates (Figure 2).

The three-storey, multi-family residential building (Figure 3 and Figure 4) was built in 1971 and is property of the Milan division of the Social Housing Agency of Lombardy (ALER). This specific case study was selected in order to validate the retrofitting approach on a large scale, from the assessment of the base conditions and the manufacturing and installation of the modular elements to the final post-retrofitting performance monitoring. The first step of the developed workflow was the materialisation of a local reference system (LRS). The main aim was to identify a unique reference system to be used both for information localisation and panel installation. In particular, a set of benchmarks were placed in positions that were evaluated as stable over time. Then, the vertex of the network was measured with a first-order total station, namely, a Leica TS30, while retro-reflective tapes (5 cm × 5 cm) were used to provide a stable orientation in the case of repositioning during different survey epochs. The redundancy of the retro-reflective tapes in stable positions also allows repositioning with the TS through the inverse intersection. The position of both the ground benchmarks and retro-reflective tapes was determined after the measurement’s least-squares adjustment. In conjunction with the network measurements, some checkerboard targets were also used. Those targets served as a tie point for scan registration.

Indeed, as previously mentioned, the production and the installation tolerances requested a higher accuracy in measuring information of the building envelope. After an extensive comparison of alternatives, a TLS survey was evaluated as optimal to derive a digital information model of the building geometry. According to El-Hakim and Beraldin [83], the key performance criteria that should be considered concerning the application requirements for the election of the proper survey strategy are:

• The geometric accuracy of the final model;
• The level of detail and spatial resolution;
• Model completeness;
• Environmental impact.

In the case of the presented case study, TLS proved to be the optimal choice. TLS can quickly collect many points (creating the so-called “point clouds”) from a few thousand to several hundreds of thousands of points per second. The resolution, i.e., point spacing, is variable across each scan since it varies according to the distance between the scanner and the object. However, with careful planning, the scan location can be approximately 1 mm (level of detail and spatial resolution mentioned above). The most significant issue with TLS point clouds is the labour-intensive process for detecting and reconstructing corner points (e.g., window areas and roof/façade area) that are also the most important aspects for panel production and dimensioning. However, an accuracy of ±4 mm for these areas can be achieved, satisfying requirements connected with panel tolerances (geometric accuracy mentioned above). By carefully planning the scan location, it is also possible to minimise shadow and obtain a sufficiently complete model. Integration of the interior information, mainly concerning the windowsills, is of primary importance. Documentation of the topography of the existing façade is necessary to apply the EASEE solution successfully. Indeed, in real buildings, façades present many irregularities (e.g., sloping walls, unevenness) to be documented in the digital model to design the anchoring system properly.

Finally, the possibility to also acquire information about the building surroundings (neighbouring buildings, vegetation, lot borders, etc.) with TLS can be of primary importance in the planning phase (Figure 5). For example, the logistics of the construction yard can be evaluated efficiently in the design phase (e.g., accessibility with the tower crane), and the panning can significantly reduce unforeseen circumstances during the installation. For the case study presented in this paper, the TLS survey consisted of 12 scans of roughly 44 million points each to survey the building envelope, performed using a FARO – CAM2 FOCUS 3D laser scanner. The mean referencing precision, evaluated by observing residuals on the target measurements, was about 3 mm.

The geometrical survey was combined with infrared thermal imaging campaigns, conducted before and after the renovation. In particular, Figure 6 shows the high thermal losses through the building enclosure, while Figure 7 reports the overall view of the south-west façade after the retrofitting, which ensured a more homogeneous surface temperature distribution.

4.2. From as-Found BIM to the Design Phase

The adopted solution for retrofitting the opaque envelope consisted of a precast insulation panel. More specifically, the chosen solution consisted of a core of laminated expanded polystyrene (EPS), 100 mm in thickness, and two external layers of textile-reinforced concrete (TRC), 125 mm in thickness. The main advantages of the developed solutions are:

• Excellent compressive and tensile strength;
• Size flexibility, since the project foresees to cast the concrete in a modular formwork with variable position shores;
• Several different possible colour and texture finishes, obtained using mixed pigments in the TRM casting and a particular silicone matrix;
• Scaffolding-free installation, thanks to four main anchors located at the panel corners made of a new fibre-reinforced concrete (FRC) system under evaluation for patenting. The advantage of this anchoring system is that each panel can be put in position by using a crane without scaffolding.

During the panel design phase, particular attention was paid to the building façade details that are commonly more critical (e.g., joints between panels and windows, balcony interfaces), in order to ensure the airtightness of the enclosure as well as the proper tolerances according to the different materials installed, which might otherwise cause thermal bridges, cracks, and detachments. Such potential issues were first assessed through finite element analysis and subsequently solved by sealing the joints with a curtain cord polyethene coupled with an acrylic silicone sealant (service temperature range −50/+150 °C, elongation at break 220–290%).

Before being actually implemented in the case study building, the resulting prefabricated system proposed was analysed through specific 2D and 3D BIM models (Figure 8). As previously mentioned, BIM is a widely accepted method for managing new constructions over time. Thanks to modelling tools and BIM libraries, it is possible to update specific building elements such as walls, roofs, floors, and windows, avoiding redrawing. Once the BIM was created, it was possible to insert different physical and material information into each 3D object, making the geometric model an “informative model” with specific detailed features.

On the other side, BIM still has some operative weaknesses, referring to rehabilitation projects on existing buildings. The use of 2D drawings for the generative process of BIM, in fact, most of the time cannot support the digitalisation of surveyed structures and related irregularities, leading to a model that does not fully correspond to real geometric conditions.

To that end, the present study developed a specific methodology to integrate free-form and parametric modelling in order to cope with the limitation of BIM software since the on-site surveys highlighted, for instance, a 7 cm out of plumb in the south-east façade of the above-presented case study. The resulting models were subsequently validated by the monitoring campaign results following a scan-to-BIM process (Figure 9), whose main goals can be summarised as follows:

• Integrating information from existing 2D technical drawings;
• Integrating existing irregularities and complex geometries;
• Identifying potential issues to inform the design;
• Identifying anchoring positions.

In particular, the integration of the slicing technique and NURBS interpolation algorithms led to the creation of a scan-to-BIM model from different data sources (photogrammetry, laser scanning, existing drawings, stratigraphic analysis) and the transfer of a proper as-found BIM to various types of analysis (clash detection analysis, construction site and BIM computing), improving the project control quality through an as-designed BIM to simplify the construction building sequence. Thanks to the 3D survey and the application of novel grades of generations [84], it was possible to reach a geometric control of all four of the building’s façades. Several tests were conducted to ensure all the irregularities for each façade were considered in an accurate digital representation such as loggias, irregular architectural details, wall surface degradation, external reinforcement bars, string courses, recesses of windows, and external MEP and stairs (Figure 10).

Simple elements characterise BIM libraries (e.g., walls, doors, windows, roofs, ceilings), where each BIM object has parametric functions that allow the user to directly edit the geometrical dimensions such as height, width, and thickness. This benefit represents the essential difference between BIM and traditional 2D digital drawings, offering a substantial technical advance based on a digital tool able to avoid redrawing and the association of different types of information such as the wall stratigraphy and physical and mechanical characteristics of materials with BIM.

Accordingly, the BIM generation of the multi-level residential building had to face the geometric irregularities of four external façades. The external wall perimeter of the building seemed to be completely regular. Thanks to the TLS survey, a pronounced out of plumb in the south façade was intercepted. This geometric condition was very important for the rehabilitation project, providing an external insulation system based on concrete panels and insulating material fixed to the outer wall with an anchoring system. The integration of McNeel Rhinoceros and Autodesk Revit automated the process and produced the correct 3D graphic representation much more quickly, skipping from AF-BIM to AD-BIM (Figure 11). The proper application of the phasing tool of Autodesk Revit was allowed to avoid long modelling procedures and exclude any vertical and horizontal overlap of individual covering elements.

The step from as-found BIM to as-designed BIM needed different modelling applications such as Autodesk Recap (post-processing), Autodesk Autocad (geometric primitives’ extraction), McNeel Rhinoceros (NURBS generation), and Autodesk Naviswork (clash detection and construction management) to accurately support the design and the installation process. In recent years, the research has moved to a targeted implementation of NURBS interpolation to create the most complex shapes of any detected artefact. NURBS free-form modelling based on GOGs 9 and 10 has made it possible to achieve very high LODs in BIM applications and significantly reduce the BIM generation, creating complex surfaces able to represent complex wall surfaces.

The results obtained from applying the proposed method support both the panel design and the tracking of the anchoring system. A second problem related to the generation of as-designed BIM and the assembly system was determining each anchor’s exact extension to maintain the insulation layer’s vertical axis despite the non-orthogonal wall. Therefore, the transition from AF-BIM to AD-BIM had to guarantee high modelling flexibility to find a proper 3D configuration. Consequently, this issue required an AD-BIM model composed of new as-designed BIM objects, such as metal anchors (with different extensions), new panels (able to change its modularity), an integrated traditional insulation system for the roof parts, window replacements, flashings, closing systems, and new connections of the external fluvial, to update and verify the model quickly in the different project phases (Figure 12).

4.3. Construction Site and Panel Installation

The Italian case study building envelope was entirely retrofitted within the EASEE project, and for the entire duration of the site work, it was ensured that all operations were conducted in absolute safety. First, the panels were moved to their final position by an operator on an aerial platform. In a similar way, the joints and interspaces of each panel line were then sealed in order to minimise convective air movements.

The retrofitting work on the Italian case study (Figure 13 and Figure 14) lasted in total less than three months (for reference, the widest façade of the building took approximately ten days), and generally speaking, the occupants expressed high interest and positive attitudes towards it. In particular, they were very satisfied with the quick scaffolding-free installation of the panels, allowing them to perform everyday activities in complete freedom.

5. Digital Twin and Cloud BIM-XR Platform Development

5.1. BIM-IoT Sensor-Based Data Access via XR: Management of Climatic Data in the 4D Living Space (or 4D Environment) That Is Easy to View

BIM, born as a control tool for the project implementation and management process, is underused due to a complete “interactivity” gap in accessing data. The massive development of IoT and connected sensor applications generates an increasing amount of data but does not fully exploit the potential to generate new knowledge, such as that related to 3D visualisation, especially of complex parameters such as energy ones.

The HOMeBIM liveAPP project intends to maximise the potential of BIM understood as information models, based on a correlated generation and management system between geometry (Object Model) and physical data from different sources (Information) (Figure 15). To reduce the interoperability gap towards software better integrated in the energy supply chain, reconstructing some missing links in the chain to exploit the wealth of information entering and leaving towards user-friendly environments that XR platforms can consult simply and interactively by end users (inhabitants, building managers, operators) represents one of the first objectives of this research. The built BIM described in the previous paragraphs absorbed a significant amount of energy to support the design and construction phase, but now it risks remaining in the drawer if gaps and barriers are not removed in a sense summarised above.

The HOMeBIM liveAPP project aims to:

• Enable the final user (inhabitant) to know the characteristics with which the environments have been designed (in this case energy efficiency and heat, but then they can be scaled with additional modules as in the case of lighting, for example) and their functioning, as well as the environmental parameters, increasing the degree of control of them, from the point of view of improving comfort, use, costs, behaviour, and good practices;
• Encourage an inclusive process of inhabitants and operators (building manager) to support the growth of awareness necessary for the achievement of the H2050 zero-energy building objectives by connecting them to the flow of data from the technical energy supply chain managed through BIM;
• Become an example of a simplification process for home automation (live app) using complex systems (BIM) with multiple applications and advanced sensors (IoT), and the field of accessibility for people to view data, allowing the improvement of habits and user adoption of good practices as well as control of consumption, costs, and comfort parameters;
• Improve the technical chain made up of expert users/consultants (building/energy manager) of energy efficiency, providing forecasts and control of energy consumption according to forecasted and monitored weather conditions and consequent optimisation of resources;
• Support the development of the HOMeBIM live app, an informed process manageable by each resident using standard devices (i.e., a smartphone) through interactive and straightforward navigation of the indoor environment, displaying the indicators coming from the intelligent system in the 4D virtual reality of the house (comfort, costs, energy consumption, environmental parameters) through immediate colours (red, green, and yellow).

This makes it possible to make the areas of improvement, comfort, costs, and energy consumption accessible, even to inhabitants with limited mobility (health), with the possibility of management and control by third parties of the environmental conditions inside the buildings according to the needs of the inhabitants, with particular attention to those who need special conditions (infants, elderly, or disabled people).

5.2. Scan-to-BIM Process Meets Computer Science Development and APIs: Development of Cloud-Based BIM Platform and Digital Building Management via a Digital Twin

In the digital age, visual communication has proven successful in displaying the products, projects, and associated information in the best possible way. Professionals in the construction sector use web solutions to communicate an immeasurable amount of information and images relating to their work. These types of data help communicate almost all the project characteristics, from the most complex BIM drawings and representations to calculations, masonry stratigraphies, and all those products connected to the design process of architects, engineers, and builders involved in the process. On the other hand, all these digital outputs have the same characteristic: they are not dynamic but relatively static information. The concepts of interactivity, immersion, and monitoring of a building are not, for the most part, invaded by digital solutions capable of communicating the behaviour of the building in real time. The time factor is not considered.

Consequently, the digital version of the building does not precisely reflect reality in terms of energy and climate. In recent years, thanks to the sharing of web platforms such as Autodesk Forge and Revit API, it has been possible to download, analyse, and use a large number of APIs in order to go beyond the current offer and address IT developments oriented to new models of information sharing associated with BIM, moving from models and static information to interactive virtual objects, 3D in-browser, and DTs to XR. Forge is Autodesk’s cloud development platform—a set of web service APIs that allow professionals to develop innovative cloud-powered applications, automate processes, connect teams and workflows, or visualise digital data. The Forge platform allows users to create and share DTs, visualise and analyse data rendering photos to 3D, integrate AR and VR, create product catalogues and configurators, automate design processes, and unlock design and engineering data to improve collaboration and build new services to address connected users (Figure 16). The main Forge solutions to support IT development are: BIM 360, Data Management, Data Visualizations, Design Automation, Model Derivative, Reality Capture, Token Flex, Viewer, and Webhooks. In particular, the development of the XR web platform made use of reworking a large number of Data Visualizations and BIM 360 APIs with the aim of:

• Visualising assets over time: additional React UI components such as timelines, dashboards, and graphs were used to complement the data displayed in the AB model.
• Visualising sensor data: sensors represented IoT devices in the AB model and were displayed as 3D icons.
• Allowing real-time or historical visualisation of sensor data in the 3D design: thanks to the Data Visualization Toolkit for IoT, it was possible to share insight into overall building operations, assets, temperature, and CO2. It was also possible to incorporate heatmap and sensor data into the as-found BIM model.
• Using models to make better business decisions, providing real-time data on assets and occupant behaviour and experience; by combining the Data Visualization extension with the sensor data, the XR web-based platform was not only able to generate current and historical visualisations, but also able to improve the understanding of how the external factors can impact overall within the environments.
• Sharing heatmaps overlaid on surfaces within the AB model to display the room temperature, model parts, and other spatial information.

Furthermore, the BIM 360 Account Admin API automated setting up projects, assigning project admins, managing members, and synchronising data with external systems. The API offered powerful search tools to retrieve specific sets of assets and other components. In particular, the BIM 360 Document Management API allows users to access, upload, and share 3D BIM models, drawings, and other project documents to maximise collaboration and share a proper hierarchy of building areas in the research case study.

5.3. Development of a Series of Living Comfort and Energy Consumption Models Based on Data from Low-Cost Sensors: From as-Built BIM and a DT to Novel Levels of Interoperability

A core component of the HOMeBIM system concerns the development of a series of living comfort and energy consumption models based on data from low-cost sensors. The data collected by these sensors will then be correlated with data derived from indoor environmental modelling and meteorological data. The modelling phase aims to develop simulation techniques and scenarios based on a series of user profiles capable of predicting the state of comfort through the control of a series of detected parameters. This will allow the correlation of these data with those relating to the management of energy resources and plants for a simulation of the effects of the choices commonly made by the inhabitants. The system architecture designed allows the monitoring and management of the environmental conditions inside the buildings according to the real needs of the inhabitants, with the possibility of optimising costs and energy demand, allowing the forecasting of energy consumption according to the forecasted conditions and monitored meteorological conditions, as well as the consequent optimisation of resources and comfort improvement behaviours, with particular attention also paid to those who need particular conditions for controlling these parameters from outside (sick, newborn, elderly, and disabled people). The monitoring system installed in the garden aims to monitor the microclimatic conditions of the project site. It consists of sensors capable of detecting multiple variables and communicating them to the data storage and management system at predefined time intervals. The monitoring system installed inside the apartment consists of numerous sensors located at specific apartment points and can detect the user’s thermal comfort conditions. The result was obtained by identifying specific descriptive fields and identification codes within the BIM project that are transmitted through the use of the IFC exchange format; thanks to the creation of parametric objects called “Sensor”, “Room”, and “Floor” corresponding, in turn, to the hardware installed directly on site, to the rooms that house the sensors, and the reference plane of the building, it was possible to improve the automatic identification of these objects within the format structure, improving the cloud platform identification of “node” objects able to connect the data flow coming from the sensors to the digital model. In particular, it was decided to identify an identification code able to relate to both the BIM application (Autodesk Revit) and the cloud platform.

Thanks to the creation of parametric objects (renamed “FLAT # THIRD_FLOOR_ROOM_01 # SENSOR_01”, “FLAT # THIRD_FLOOR_ROOM_01”, and “FLAT # THIRD_FLOOR”), it was possible to export the BIM models in their native format (RVT, RFA, RTE, RFT) and convert them, in turn, into IFC 2x3 format using the following diagram:

• IFC 2X3 Coordination View 2.0;
• File type: IFC;
• Phase to export: default phase to export;
• Space boundaries: none;
• Project origin: current shared coordinates;
• Property sets: export IFC common property sets;
• Level of detail: low, medium, and high;
• Advanced: export parts as building elements, allow the use of mixed “solid model” representation, use active view when creating the geometry, use family and type name for reference, and export the bounding box.

This scheme made it possible to define a viable line of development/creation/conversion of the BIM-to-cloud model, facilitating the uploading of the model and the reading of information and physical/mechanical parameters of the materials associated with each parametric object, and favouring the analysis and reading of real-time data from the monitoring system installed on site (Figure 17).

The web platform was also programmed for a more user-friendly and portable display. Thanks to specific APIs, it was possible to orient the traditional visualisation via the web browser to the mobile version, thus facilitating the monitoring of the building directly via a mobile phone or tablet (Figure 18).

6. Discussion

A monitoring campaign made it possible to acquire precise data to calculate the thermal transmittance of the external walls during the retrofitting process. The precise measurements were made with thermal flowmeters, requiring a period of acquisitions every 15 min.

These measurements made it possible to

(i)

Obtain the thermal transmittance of the building envelope and compare it with the theoretical value calculated during the design phase;

(ii)

Analyse the internal spaces’ environmental conditions;

(iii)

Detect the thermal flow between inside and outside and the surface temperature;

(iv)

Detect the temperature of the internal environment and the relative humidity;

(v)

Install a data logger for the management of the collected data;

(vi)

Measure the amount of incident radiation, detecting the temperature of the external air;

(vii)

Verify the system’s actual behaviour in terms of surface temperatures and thermal transmittance. The measuring instruments were placed on a wall portion of the housing representative, facing north-east to avoid direct solar radiation;

(viii)

Verify the homogeneity in surface temperatures and the absence of system plant components (thermographic survey).

Thanks to the measurements, it was possible to obtain a thermal transmittance of 0.270 W/m²K, thus highlighting how the overall behaviour is positively influenced by the presence of the interspace set (panel–wall) both in winter and in summer. In summer, temperatures are attenuated, while in winter, the average of all environments is 6 degrees higher. The energy consumption measured for the last five years is reported in Figure 19. The energy consumption for the winter of 2015–2016 was reduced by 36%, with an energy saving of about 52 kWh/m²year with respect to the previous winter season. Considering the dependency of the energy consumption on the local climate condition, different heating seasons were analysed (Figure 20). The mean energy consumption before retrofitting was equal to 187.65 kWh/m²y, which is 45% more than the energy needed after the external walls’ retrofit [85].

7. Conclusions

Deep renovation of the existing building stock can significantly contribute to reducing greenhouse gas emissions, as seen in the European, national, and local energy policies. In this context, this article presented a potential solution for the renovation of external façades developed within the EASEE project, detailing its application in an Italian case study. The results of that renovation project successfully prove the effectiveness of such a novel modular prefabricated system, whose key features are a high insulation potential, durability, and wide flexibility in terms of colour and texture finishes.

The project allowed the authors to verify and test a novel façade system and the proposed installation process, showing a high innovative potential regarding the ease of installation, its performance, and the overall quality and replicability.

Furthermore, the process automation levels were favoured by the application of GOGs 9 and 10. The GOGs increased the level of interoperability between different project phases, moving from as-found to as-designed and as-built projects. The scan-to-BIM process and the creation of a DT made it possible to manage the life cycle of the redevelopment project by appropriately managing the morphological and typological aspects of the building. In particular, in geometric and metric terms, the applied process made it possible to manage the digital copy with a degree of GOA DI 1.5/2 mm. The creation of custom objects as-found from the 3D scans allowed the authors to maintain high LODs simultaneously without running into a loss of information. A bidirectional object–information relationship was also implemented thanks to the creation of BIM sensor objects equipped with interactive links within the DT.

In addition, the know-how of this research has been transformed into a digital tool able to support the sharing of complex and simple models from point clouds. The experience gained was crucial to understanding the programming logic shared by Autodesk through hubs such as Autodesk Forge and Revit API docs. Several tests were conducted to test the limit of APIs in terms of development. IT development was supported by open-source libraries of API docs which allowed identifying, orientating, and improving tools in the software architecture. Once converted, the DT can be easily viewed and shared via the XR web platform, helping professionals and non-expert users to verify the building behaviour. The overall approach has high potential regarding the reliability of the data fluxes from the geometrical survey to the retrofitting design phase and the real data monitoring integration. The actual main barrier, according to our experience, is related to the continuous monitoring and, in more detail, to the cost and the maintenance of the sensor network that might discourage integration in actual construction practice of the overall approach from both the final user and potential investor points of view.