Next Article in Journal
Numerical Analysis of Pillar Stability in Longwall Mining of Two Adjacent Panels of an Inclined Coal Seam
Next Article in Special Issue
Formation of Rational Sets of Machines for Excavation Work in Urban Areas
Previous Article in Journal
Digital Calibration for Gain, Time Skew, and Bandwidth Mismatch in Under-Sampling Time-Interleaved System
Previous Article in Special Issue
Research on Optimization Design of Geometric Parameters of a Novel Frame-Embedded Track (NFET)
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Analysis of Design Change Mechanism in Apartment Housing Projects Using Association Rule Mining (ARM) Model

1
Department of Architectural Engineering, Hanyang University, 222, Wangsipri-ro, Sungdong-gu, Seoul 04763, Korea
2
Department of Architecture, Kangwon National University, 346, Jungang-ro, Samcheok-si 25913, Korea
*
Authors to whom correspondence should be addressed.
Appl. Sci. 2022, 12(21), 11036; https://doi.org/10.3390/app122111036
Submission received: 29 September 2022 / Revised: 27 October 2022 / Accepted: 28 October 2022 / Published: 31 October 2022

Abstract

:
Apartment housing occupies the highest proportion of the domestic construction market and significantly influences the flow of the real estate market. Frequent design changes and reconstruction in new apartment housing projects lead to an increase in construction cost and schedule, and a decline in design and construction quality, which is an important issue affecting the quality of use for occupants. The causal relationship of design changes and error in new apartment building projects has not been previously identified. Accordingly, design changes management activities in the construction phase using reactive manner are a critical risk that causes the productivity of the project to deteriorate. In this study, a complex and non-linear causal relationship between the design change factors was investigated using the association rule mining technique (ARM), a type of data mining technique. In particular, the associated relationship between design change factors that can be changed according to conditions that significantly affect the productivity and performance of projects, such as a contractor’s ranking in the field, contract price which means the project size, and contractor selection methods, was identified. The association rule between the design changes at the construction phase derived in this research can be used as a guide to identify and minimize the risk of design changes in advance.

1. Introduction

Regardless of the public and private sectors, until recently, the co-planned ordering method was used as the main procurement system in design [1]. In a DBB (Design-Bid-Build)-type construction project, the designer selected by the client delivers the design results without considering the constructability, which causes numerous design errors and changes in the construction phase [2]. The design documents and deliverables created in the design stage may contain several design errors, omissions, overlapping, and inefficient design proposals, which present potential risks that can negatively affect the construction period, cost, and quality [3]. However, this occurs because nobody is responsible for verifying these issues at the construction stage or contractually supplementing them.
Meanwhile, apartment houses occupy the highest proportion of the construction market and significantly influence the flow of the real estate market in South Korea. In fact, the apartment market acts as an important factor that determines the economy of the entire construction industry. Therefore, most construction companies sell apartment houses as their main products, but in reality, it is also the type of project with great loss due to numerous errors of design and construction, re-design, and re-construction [3].
Especially, the frequency and risk of design changes in the apartment housing projects vary depending on different internal and external environmental factors, such as the project size, client, performance of participants, construction cost, and difficulty, which define the characteristics of project. Design change risks should be considered a critical element by managers [3].
However, existing relative studies have not identified the causal or precedent relationship between various design changes or design error factors that cause quality deterioration in new apartment building construction. In addition, the analyses of the characteristics of each factor, the risk level, and frequency of occurrence are limited. Considering the design change factors of apartment housing can occur in a chain-like manner by a complex and non-linear causal relationship between the causes, it is important to analyze the association rules between each design change factor through a quantitative methodology [4,5]. It is very effective to apply data mining methodology, which is used to analyze the relationship or correlation between items for marketing strategies, to find out the mechanism of design change.
Therefore, this study intends to conduct an analysis using an association rule model (ARM), which is one of the data mining methodologies using unsupervised learning, between the design change factors that occur in the construction stage, targeting a new apartment building project. Using ARM, an analysis of the design changes according to project conditions can identify the root causes of design change factors that have a fatal impact on productivity management, such as cost, time, and quality of the project, and prevent its occurrence in advance. In addition, it can provide critical information and data to the project manager in advance by predicting the possible design changes or errors based on the design changes that have occurred.

2. Literature Review

2.1. Impact and Correlation of Design Change in the Apartment Housing Projects

Under the current law of the Republic of Korea, a design change refers to the amount of construction increased or decreased due to revisions and supplementations of design documents that occur during construction [6]. The design changes during construction occur due to the following possible scenarios: (1) the contents of the design document are unclear, with omissions, errors, or contradictions between them, (2) the condition of the site is different from that in the design document, (3) the reduction of construction costs and shortening of the construction period by using new technology and construction methods, which has a remarkable effect, and (4) when the ordering organization requires a change in the design drawings [6]. Especially, (1) and (2) occur when the construction stage is started without noticing that the design has many errors and omissions. It is an uncertain risk that impacts productivity, such as construction cost, schedule, and quality.
Design change risk is a major factor that makes construction projects more uncertain and sensitive to changes compared to other industries [4]. Almost all apartment housing projects require modifications to the design or scope of work [5]. In many cases, the requirements of the client are not clearly confirmed at planning phase, because of the review and verifications regarding the feasibility and planning [5,7,8,9]. Moreover, the business plan can be inevitably changed, unlike at the time of the contract [5,7,8,9,10]. In addition, design changes may occur because the drawings and specifications delivered to the general contractor (GC) at the construction stage were different from the actual site conditions, or there were problems with the design quality [6].
Existing studies have argued that the three main criteria for a successful project are time, budget, and quality, which are all inevitably affected by design changes [11,12,13,14]. The important factors that cause failure in the construction industry were organized into 62 factors from various perspectives [15]. The top 10 factors were then extracted, with frequent design changes identified as a major factor [15]. In addition, 17.43% of cost fluctuations in construction projects are due to various factors, the most important of which are the design and specification changes [16].
Identifying the cause and influence of these design changes is a very important management target of construction manager, because that can cause a decline in productivity of project [17]. Therefore, it is important to study the type of change, the cause of its occurrence, and the ripple effect. Design changes occur due to various causes, such as site conditions, civil complaints, and design errors [18], and the cause and type of design change are the most important factors in a design change analysis [19]. Apartment houses are constructed in the same or similar space of the entire floor plan, and these individual buildings are arranged to form a large district. Thus, the occurrence of design changes in apartment houses is a critical risk that can lead to numerous consecutive design changes and errors [3]. In other words, as individual design changes can cause chain occurrences in other locations or materials such as public facilities, underground, underground parking lots, balconies, and rooftops, the influence according to the type of design change must be considered [20].
According to a study that analyzed the relationship between the cause and effect of design change in the construction project, factors such as schedule change, workload change, and design quality decrease are the most important causes of design change. Therefore, continuous communication and development by stakeholders participating in the management of drawings and design documents are important areas. Thus, increased communication between stakeholder groups and continuous learning through project experiences are important strategies [21]. A system dynamics model was also developed to capture the dynamics of the design changes and errors and systematically evaluate their negative impacts. According to this model, a project schedule was considered significantly delayed due to design errors, despite the manager taking continuous schedule recovery actions [22]. In addition, based on the simulation model developed to confirm the project performance in terms of the total project duration and expected amount of rework, the shortening of the project period and the expected amount of rework was determined to vary according to the accuracy of the initial information and the sensitivity of the project activities [23]. In addition, as the leading cause of the design change in the construction stage, the customer demands and expectations for quality were selected [24]. Although existing studies have provided a comprehensive understanding of the causal mechanism between design change factors, the design issue remains an individual analysis topic that does not consider the elements of the design issue in an integrated manner. In particular, understanding the flow of design changes is an important area of construction management that can manage cost, schedule, and quality. In fact, for construction management, research, deriving the relationship between building defects and strategies for comprehensive management was also conducted [2]. In order to investigate the association rules and probability of occurrence between building defects, a study was conducted applying the machine learning method that combines ARM and Bayesian network. Through this, it was found that the higher the probability of the derived high-risk defect, the lower the inspection score and construction quality. In addition, the dynamic causal relationship between defects was identified based on the time series information [25]. A study investigating the relationship between the defect type and quality inspection grade of public construction projects using the ARM model and fuzzy logic was also conducted. In particular, after identifying the association rules between each defect using ARM, multiple defect types were derived through K-mean clustering and ANOVA analysis [26].
As such, it has been proven that data mining can be applied effectively to qualitative and irregular information of the construction industry and projects. This means that it can also be used to deal with the main subject of very qualitative and unspecified design change in the construction industry. Thus, data mining techniques can also be used to handle design change data which is the major issue in the construction industry. However, unlike the defect data of buildings, the design change data are based on the information in the drawing, so they do not have time series information or numerical data that express the risk of design change.
Therefore, the purpose of this research is to investigate the mutual association and generation mechanism of design change issues occurring in new apartment housing projects. In particular, project conditions, such as capabilities of contractors, contract price, and bidder selection methods, are main factors that greatly affect the productivity of the project. Therefore, it is also necessary to investigate whether the mechanism of design changes actually changes depending on the project conditions.

2.2. ARM Methodology

The ARM is a process of deriving a statistically significant relationship with a specific event, article, or item in the database [27]. ARM has traditionally been developed to analyze distributors’ POS data based on sales and product information and is most widely used in the marketing field. When a customer purchases an item at the checkout counter of a store, customer information and purchase item information are input together, which is called market basket data, and the purchase item information included in one customer, that is, one transaction. There are many forms of transaction in basket data, and the Association Rule Mining technique is to find the association and pattern of transaction through this large amount of data. This associated rule analysis model is used by product sales companies to establish marketing strategies by predicting which products customers who purchase specific products will purchase together. It can also be used to change the store display method in connection with customer movements, or to develop package products.
From this point of view, this study intends to apply ARM, a data mining technique, to identify the relationship between design change items that occur during construction work and the mechanism of their occurrence. By applying the ARM model to this study, comparing “market basket data” and “design change data” of apartment housing project, “one customer” can be applied as “a project” and “purchase product information” as a “design change item”. Through this, it is possible to find the association and pattern between design change items that occurred in multiple projects. In particular, it is possible to increase a certain pattern and reliability of the occurrence of design changes by analyzing a large amount of design change data of a new apartment building project having a similar shape, scale, plan, and construction method.
Using the ARM approach, it is possible to discover meaningful rules among items by the deriving ‘condition-result (if-then)’ patterns between items [28]. This if-then can be defined as the relationship between the conditional part of the data group and the conclusion part rather than the meaning of the “if-then” in logic. Essentially, the purpose is to identify events with a high probability of occurrence by finding hidden patterns in the data group. In association rules, scales such as support, confidence, and lift are used to select meaningful rules from among the patterns.
-
Support: Support for two items x and y indicates the ratio of the number of data including both item sets x and y among the total number of data. Support is used as a basis for determining good rules (high frequency or high composition ratio) or reducing unnecessary operations (Pruning).
Support(X → Y) = 𝑃(X ∩ Y)
-
Confidence: The proportion of data containing item Y among data with item X (conditional probability). A more reliable rule is more likely to be useful.
Confidence(X → Y) = 𝑃(X|Y) = 𝑃(X ∩ Y)/𝑃(X)
-
Lift: There is a limit to discovering a meaningful association rule only with support and confidence, and it is measured to discover new rules. Lift is expressed as the ratio of the rule confidence to the rule expected confidence. In other words, the lift index generally indicates the dependence between the items in the item set and can be an index indicating the strength of the association rule. If the lift index is greater than (positive relationship) or less (negative relationship) than 1, it indicates that it is better than a random chance. If they are independent, lift = 1, and the expected confidence of a rule is defined as the value obtained by dividing the value obtained by multiplying the support of the left and right sides by the support of the left side.
Lift(X → Y) = 𝑃(Y|X)/𝑃(Y) = 𝑃(X ∩ Y)/𝑃(X)𝑃(Y)
The three aforementioned scales are important criteria for adopting derived association, such as minimum support, minimum confidence, and minimum lift. Determining the threshold has a decisive influence on rule adoption. The analysis results of the associated rules are derived in the form of {design change X} => {design change Y}. This means that the rule meets all three thresholds of the scale, such as (1) the proportion of projects with both X and Y design changes; (2) proportion of projects with Y out of all projects with X; and (3) the value multiplied by the probability of occurrence of X and Y divided by the support of X. Therefore, the rule {design change X} => {design change Y} can be interpreted as meaning that if design change X occurs, Y is more likely to occur, or if design change Y occurs, design change X is more likely to occur.

3. Data Collection and Pre-Processing for ARM

The data used for ARM between the design change items indicate the design changes for a new apartment housing project conducted by a major public client. The original raw data are shown in Table 1. The data consist of 35,194 design change issues that occurred in a total of 517 projects. All design changes occurred in 12 disciplines (including architecture, structure, and mechanical) and for 17 reasons, as shown in Table 1. In addition, the attribute information of design change issues consists of the contract name, contractor, construction cost, and contractor selection method.
Based on the 35,194 design changes listed above, data pre-processing and purification were performed, and a total of 6323 design change items were extracted. To this end, by analyzing the raw data, items with missing attribute information such as contract name, main contractor, initial contract price, completion date, discipline, reason for design change, and details were first filtered. During the secondary filtering process, duplicate items and invalid attribute information were removed. Table 2 presents the data structure used for the ARM analysis through data pre-processing.
As described in Table 2, the dataset consists of 6323 design change data generated from a total of 299 projects and have 9 attribute details including number, main contractor, and contractor selection methods. All design changes occurred in 11 disciplines and occurred in 28 locations or materials of apartment house, including PIT floor, living room, dining room, corridor, and so on. Finally, there are a total of 10 reasons for design changes, including extension of schedule, civil complaints, sales promotion, project plan change, and so on.
The individual design change data, input to the ARM analysis, consist of three attribute information, such as discovery, reason for design change, and location/materials, among the data organized in Table 2. Derived results of the association rules are expressed in the form of {discipline, reason for design change, location/materials} => {discipline, reason for design change, location/materials} and have the meaning of “if” => “then”. In this rule, the left part is expressed as LHS, and the right part is expressed as RHS. Information other than the three attributes, such as region, main contractor, contract selection method, contract price, and detailed reason, was used as a basis for interpreting the possibility and meaning of rules derived from the analysis result of related rules. In particular, three items, the main contractor, contract selection method, and contract price, were used as criteria for grouping to find out how the association rules of design change are derived differently for each project condition.

4. Research Approach

This research was conducted for the purpose of analyzing the behavior and association of design changes occurring in new apartment housing projects, which are the most frequently ordered delivery forms in South Korea. In order to analyze the relationship between these design changes, the research process as shown in Figure 1 was constructed. First, in the research backgrounds, the risk of design changes and errors occurring in the current apartment housing project was mentioned, and the limitations of existing studies to solve this were analyzed. Through this, the importance of the ARM model applied in this study was derived. Second, the necessity of this study was derived through literature review. To this end, studies that identify the causes or risk of design issues currently experienced in the apartment housing project were analyzed, and the results, directions, and limitations of these studies were analyzed. In addition, the justification of the study was secured by differentiation of the existing studies in this study. Third, data acquisition, purification, and input data for ARM analysis were constructed. is the data are a list of design change items of the apartment housing project acquired by the public order in South Korea, and are processed into a structure for ARM analysis. Finally, ARM analysis was performed and the results were interpreted. ARM analysis was performed by dividing it into an analysis based on all case data and an analysis by project conditions. Through this, the association and chain of occurrence patterns between design changes in the apartment housing project were identified, and in particular, the occurrence patterns of design changes that vary depending on the contractor’s ranking, contract price, and contract selection method were identified.
Under the research flow of Figure 1, the ARM model as shown in Figure 2 below was also designed to optimize the verification of the association between the design change data.
Association rules were derived from the pre-processed design change issue data using the prior algorithm. The association rule was first adopted by filtering the concurrent items that satisfy the threshold values of each criterion, such as support 0.1, confidence 0.3, and improvement 1.0. All of the rules adopted first were categorized into four groups, as shown in Table 3. “●” indicates rules that are known and easily predictable by anyone, and there is no need to clarify them through this study. In addition, rules indicated by “X” were derived through data mining, and it is an unrelated and irrelevant rule that occurs by chance or by the nature of the case data group. Conversely, the rules marked with ◌ and △ indicate new rules that were previously unknown, and can be identified owing to academic information and recent studies and technologies; namely, they are rules worth investigating more. Among these, a set of rules that were previously unknown, such as “◌” and “△”, which can be sufficiently explained or related, were newly identified through this study.

5. Association Rule Mining between Design Changes in the Construction Phase of Apartment Housing Projects

5.1. Results of Deriving Association Rules between All the Design Change Data

Regardless of the project characteristics, ARM was implemented based on 6323 total design change data to determine the association rules between all the design change issues of the new apartment building construction. Table 4 demonstrates the top 22 of the total association rules created for the design change issues that occurred, in order of highest support. There was a total of 22 rules that satisfied the thresholds of support, confidence, and lift. The derived association rule number 1 is {architecture, design change, underground parking lot} => {architecture, design change, public facility}, with a support rating of 16.4% and confidence of 68.1%; that is, both design change items, {architecture, design change, underground parking lot} and {architecture, design change, public facility} occurred in 16.4% of the projects. This indicates that there is a 68.1% probability that the {architecture, design change, public facility} design change will also occur in the project where the {architecture, design change, underground parking lot} design change occurs. Considering the properties of Rule 1, it indicates that the design changes of the construction type of the underground parking lot and public facilities are mutually related. It can be presumed that the design change of the MEP (Mechanical, Electrical, and Plumbing) of the underground parking lot causes a design change in the construction type, such as finishing. Therefore, it can be assumed that the MEP of public facilities is also changed in a chain occurrence, thus the type of construction also changes. If we filter the issues of {building construction, design change, underground parking lot} and {building, design change, public facility} from the 6323 total original data, 354 items are derived. The reasons for their design change were as follows: (1) design error, (2) estimate error, (3) voluntary change due to anticipation of defects, (4) quality improvement, (5) design improvement, and (6) cost reduction. Rule 2 is “{earthwork, site conditions, other} => {earthwork, site conditions, underground}”, with a support rating of 15.4% and confidence of 73.0%. In other words, {earthwork, site conditions, other} and {earthwork, site conditions, underground} design changes occurred in 15.4% of the projects, indicating that the probability of design change is 73.0%. If the design change items with civil works, site conditions, underground, and other attributes are filtered, a total of 437 datapoints is selected. Among these, the reasons for the subcategory of design change for items whose design change is due to site conditions are “earthwork, foundation (pile, bearing capacity) settlement, temporary facility change, site condition change, construction method change, civil complaints around the site, design error, permission request, quality improvement, and defect prediction”.

5.2. Results of Deriving Association

In particular, the design change of the construction type due to a voluntary change owing to the demands of government agencies and headquarters, as well as the changes in sites, errors, quality improvement, and civil complaints, simultaneously occurred with the design change of other construction types and parts. This indicates that a design change in the construction industry may cause other design changes.
Based on the derived association rule, the significance of the rule was analyzed and classified, as shown in Table 4. Rule 20: {architecture, site conditions, etc.} => {earthwork, site conditions, underground} does not have a direct relationship at first glance. However, considering the detailed reasons for the {architecture, site conditions, other} items, most of them are changes in conditions outside of construction, such as change of materials to be paid, settlements, change of business approval documents, and safety inspections. Therefore, the type of underground civil works may also be changed depending on the site conditions. In fact, regarding the detailed change factors of the {earthwork, site conditions, underground} items, quantity settlements, and changes in the construction methods are the major reasons, which may be affected by the {architecture, site conditions, other} items owing to reasons such as business approval documents, change of materials to be paid, safety inspections, etc.

5.3. Rules between Design Change Data by Project Condition

The result of analyzing the association rule based on the entire design change data does not reflect that the occurrence pattern or mechanism of the design change may vary depending on the various characteristics of the project. Therefore, the data group is classified based on various factors that are sufficiently suspected to affect the design and construction quality, such as the subcontracting orders or technology, construction scale (construction cost), and method of selecting a successful bidder, to determine whether the rules of association between these design change items are meaningful. Table 5 demonstrates the data group classification criteria according to the main contractor; that is, the subcontract order of the general contractor, amount of construction contract, and successful bidder selection method.

5.3.1. Result Analysis of the Association Rules between the Design Change Issues by the Subcontracting Order of the Main Contractor

Generally, a higher rank of a contract indicates more varying design and construction quality management systems and strategies. However, it is necessary to determine whether this assumption is proven by the occurrence pattern such as the number of design changes or associations and whether the effect of management can prevent the chain occurrence or mechanism of the design changes. Accordingly, the design change data were classified by the subcontracting order of the general contractors participating in the construction stage of the apartment building project (three groups: 1st to 30th, 31st to 100th, and 101st and higher), and the associated rules were analyzed. As a result of the analysis, the association rules were derived as follows: (1) 25 projects ranked = 1–30, (2) 20 projects ranked 30–100, and (3) 22 projects ranked 100 and above. Among these, the characteristics were analyzed by filtering each of the top three rules based on the support, confidence, and lift values.
For the projects in which companies that ranked 1 to 30 participated, they demonstrated a difference in the pattern or relevance of the design changes from those in which the companies ranked 30 or less. Namely, the design change factors that made up most of the design change mechanisms were voluntary for sale promotions and quality improvement. As a result, the design changes of the related work types and parts occurred. Considering the detailed reasons, the rate of change due to the design errors was not high, which indicates that the project management capability of the main contractor performing the apartment housing project affects the occurrence of the design changes and chain occurrences. According to the results of the design change association rules for companies ranked below 30–100, it was found that design changes mainly in the earthwork field cause design changes in structures and architecture. Due to the change of the earthwork plan sequentially progressing, outdoor piping → outdoor → corridor → balcony, etc., a practical clarification is needed in this area. The last project in which the top 100 companies participated demonstrated a pattern in which the design changes due to the design errors in multiple construction types expanded to adjacent and connected spaces and related construction types, causing design errors. In particular, the change in the earthwork plan for the outdoor space implemented at an early stage of construction demonstrated a correlation that caused the change in the construction plan in the underground field, such as earthworks. Table 6 compares and analyzes the results of the design change association rules according to the subcontracting order of the main contractor.

5.3.2. Analysis of Result of Association Rule between Design Change Issues by Construction Price

The construction scale of an apartment housing project can be judged according to the contract price, which is related to the number of project participants and manpower. In addition, for large-scale construction, companies with high subcontracting ranks often participate, thus the frequency or occurrence pattern of design changes may vary due to different project management strategies and methods. In addition, there is a risk that the scope and difficulty of management may increase, making it difficult to predict and respond to design changes. Namely, based on the contract amount stipulating the construction scale, the design change issue data are classified into the three following groups: less than 50 billion won, less than 50 to 100 billion won, and more than 100 billion won. The relative rule analysis was then performed, and the occurrence patterns of these rules were comparatively analyzed.
The design change mechanism in relatively small projects with less than 50 billion won can be considered a quality control failure due to design errors occurring in earthwork and construction projects. The reasons for the change in the overall design change items are design and quantity errors, which are estimated to create a mechanism for generating design changes and extending to completely new areas, such as underground, balconies, and public facilities. On the other hand, considering projects worth 100 billion won or more, the main reason for the change appears to be a planned change for quality improvement due to a voluntary design change at the site or head office to promote sales. The design change items included in the top three rules are spaces within a unit household, such as rooms, hallways, living rooms, and kitchens, and are more likely to simultaneously occur than causal relationships. Table 7 compares and analyzes the results of the design change association rule according to the contract amount.

5.3.3. Analysis Result of the Association Rules between the Design Change Issues by the Successful Bidder Selection Method

The ordering system of a construction project can significantly affect the productivity and quality of a project. In particular, the disconnection between the design and construction phases and the lack of collaboration among the participants performing each task frequently causes design errors and design changes in the construction phase, which is a factor of construction waste. Therefore, we analyzed the pattern of occurrence of design changes and the changes in correlation according to the successful bidder selection method, which is one of the conditions for the ordering method of new apartment building projects. For the method of selecting successful bidders, three types of screening systems, the lowest price screening system, and the turnkey screening system existing in the case data, were analyzed.
For the project where a successful bidder was selected by the qualification method, only one major construction type was included in the association rule mechanism. Namely, it was found that the design change that occurred in the construction type did not cause the design change in other construction types. However, it appears that the design change of the construction type of the underground parking lot is related to public facilities, balconies, and rooftops; thus, there is a mechanism for design change occurring only for the type of architecture discipline. Companies that passed the qualification criteria and have been awarded a bid are given the opportunity to bid in the order in which they offered the lowest price. Although the lowest price was proposed, the construction performance capability included in the qualification criteria was recognized, which indicates that it has certain project management, design, and construction quality management capabilities. In contrast, the project conducted with the lowest bid method occurred in earthwork and architecture. In particular, it was found that several changes occurred due to the failure of the design management in the common parts of the construction, such as the underground parking lot, rooftop, and outdoors, and these items were linked to the relevant part or even caused design changes in other adjacent spaces. In addition, similar to the projects conducted by companies ranked 100th or higher, several design changes occurred due to insufficient planning in the earthwork part in the early stage of construction. This has been shown to affect underground and outdoor construction. Lastly, unlike other construction projects, the turnkey ordering method is highly likely to include design change items that occurred in the electrical and mechanical construction types in the design change generation mechanism. Apparently, chain occurrences of the design changes were from the indoor plumbing to the balcony, entrance, kitchen, and living room. However, most of the detailed reasons for the design change are the headquarters and that the site performed voluntary design changes to promote the sale because the turnkey construction is generally large-scale, and there is a high possibility that companies with a relatively high contract rank will participate. On the other hand, there were not several design changes relative to architecture, structure, and earthwork through the design review and construction plan establishment of the contractors who participated from the design stage. Table 8 summarizes the results of the derivation of the association rules for each successful bidder selection method.

5.4. Discussions

As a result of analyzing the pattern of occurrences of the association rules between each item based on the apartment housing design change issue data, the following implications were derived:
  • The subcontracting order of the main contractor participating in the new apartment building project determines the quality control capability of the design and construction; the types and patterns of design changes that occur due to that are different. In addition, the failure of design change management due to general mistakes, omissions, or errors causes a series of design changes to multiple types and parts that are not directly related.
  • The form of the association rule between the design change issues changes according to the contract amount of the new apartment building project. Small and medium-sized projects of less than 50 billion won have the characteristic of causing chain design changes including underground parking lots, public facilities, and rooftops, due to the instability of the initial earthwork process plan, for which the failure of quality control is the main cause.
  • The method of selecting successful bidders, such as those who propose the lowest price, undergo qualification screening, and turnkey, is closely related to the management capabilities and responsibilities of the main participating contractors. In low-cost projects, design changes occur mostly due to basic design errors, which deteriorate quality and induce changes, whereas, in turnkey projects, design quality and constructability reviews are conducted in advance at the design stage. Therefore, design changes due to mistakes and errors do not cause a chain reaction.
  • According to the existing cases and studies related to the impact and risk of design changes, it was difficult to find a design change correlation between the parts and construction types that are not directly or indirectly related, such as underground parking lots and balconies, outdoor facilities, and public facilities; however, a relationship between these was found as a result of this study. This indicates that, as presented in the Mechanism of Design Change, the design change that occurs first can cause design changes in the same/similar work type, adjacent sites, MEPs, etc., which are interconnected.
This study was a first attempt to apply the ARM model based on qualitative design change information composed of text. All studies that have analyzed the characteristics and risks of design changes have limitations in that it is not possible to analyze the behavior or relationship between design change items directly. These studies only include research on the causes of design changes, such as schedule changes, workload, and lack of quality control capabilities, research on models that quantitatively analyze the risks of design changes and errors, and research on models that measure project performance decline due to design changes and rework in the construction phase. On the other hand, the ARM model-based design change analysis model presented in this study is meaningful in that it quantitatively identified the possibility that design changes that occur during project operation can directly cause other design changes or simultaneously. Unlike the ARM model applied to other fields in the construction field, such as building defects and project claims, it was a very difficult field to analyze the behavior and relationship between design change data with very qualitative and various attribute information. From this point of view, this research presented the first analysis model that can quantitatively identify the direct relationship between design change factors and has scalability that can be applied to various projects in the future.

6. Concluding Remarks

The purpose of this study was to investigate the relationship between design changes that occur in new apartment building construction projects in Korea. It started with the assumption that design change issues do not occur alone, and that one design change can cause another design change that was not previously known. The ARM data mining technique was applied to investigate this. Based on the design change data of the apartment housing project conducted in Korea, the correlation between these is analyzed based on the derived association rules. The previously known strong rules and rules that were not related and had no basis were excluded, and other rules were considered: (1) rules that could be explained but were previously unknown, and (2) rules that were previously unknown and may be related from an expert’s point of view, but were not easy to explain. In particular, design change patterns and associated rules according to project conditions that significantly influence the characteristics of a project were investigated, such as the capacity of the main contractor for the apartment housing project, the size of construction, and the method of selecting a successful bidder.
Previously, the relationship between design change factors was identified based on regression analysis, statistical analysis, and qualitative surveys, while the extent of how significantly they affect each other was unknown. This study is meaningful because it performed a quantitative analysis by finding patterns that were based on actual data. In particular, the reliability of the association rules derived in this study was demonstrated by analyzing a large amount of case data for apartment houses.
The main subject of this study included apartment houses, which were limited to a pattern analysis according to conditions such as the subcontracting order of the main contractor, the contract amount, and the method of selecting a successful bidder. In the future, through a more detailed analysis of the various attributes of the design change issues, it will be possible to examine the relationship between the design change occurrence patterns and issues using various methods. Moreover, a study will be conducted to derive financial risks and their risk levels of design change issues by linking the ARM model, proposed in this study, with the loss cost data due to design change.

Author Contributions

M.K. developed the concept and drafted the manuscript. J.L. reviewed and revised the manuscript. J.K. supervised the overall work. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. NRF-2022R1C1C1009927).

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Acknowledgments

The authors would like to thank the Ministry of Trade Industry and Energy of the Korean government for funding this research project.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Cheng, R.; Johnson, A. Motivation and Means: How and Why IPD and Lean Lead to Success; Lean Construction Institute and Integrated Project Delivery Alliance: Minneapolis, MN, USA, 2016. [Google Scholar]
  2. Park, I.; Kim, H.; Lee, H.; Kim, D.; Min, Y.; Cho, H. Analysis of Causes and Impact of Change Orders in the U.S. Military Construction Projects. J. Korea Inst. Build. Constr. 2021, 21, 213–219. [Google Scholar]
  3. Sacks, R.; Goldin, M. Lean Management Model for Construction of High-Rise Apartment Buildings. J. Constr. Eng. Manag. 2007, 133, 374–384. [Google Scholar] [CrossRef]
  4. Yana, A.G.A.; Rusdhi, H.A.; Wibowo, M.A. Analysis of factors affecting design changes in construction project with Partial Least Square (PLS). Procedia Eng. 2015, 125, 40–45. [Google Scholar] [CrossRef] [Green Version]
  5. Atkinson, A.R. The Pathology of Building Defects; A Human Error Approach. Eng. Constr. Archit. Manag. 2002, 9, 53–61. [Google Scholar] [CrossRef]
  6. Kang, H.; Lee, W.; Kim, M.; Kim, Y. A Case Study on the Risk Impact and Loss Cost of Temporary Aircraft Runway Construction. Korean J. Constr. Eng. Manag. 2020, 21, 32–39. [Google Scholar]
  7. Minato, T. Representing Causal Mechanism of Defective Designs: A System Approach Considering Human Errors. Constr. Manag. Econ. 2003, 21, 297–305. [Google Scholar] [CrossRef]
  8. Chong, W.K.; Low, S.P. Assessment of Defects at Construction and Occupancy Stages. J. Perform. Constr. Facil. 2005, 19, 283–289. [Google Scholar] [CrossRef]
  9. Lee, S.; Cha, Y.; Han, S.; Hyun, C. Application of Association Rule Mining and Social Network Analysis for Understanding Causality of Construction Defects. Sustainability 2019, 11, 618. [Google Scholar] [CrossRef] [Green Version]
  10. Forcada, N.; Macarulla, M.; Gangolells, M.; Casals, M.; Fuertes, A.; Roca, X. Posthandover Housing Defects: Sources and Origins. J. Perform. Constr. Facil. 2013, 27, 756–762. [Google Scholar] [CrossRef]
  11. Assaf, S.A.; Al-Hejji, S. Causes of delay in large construction projects. Int. J. Proj. Manag. 2006, 24, 349–357. [Google Scholar] [CrossRef]
  12. Ameh, O.J.; Soyingbe, A.A.; Odusami, K.T. Significant factors causing cost overruns in telecommunication projects in Nigeria. J. Constr. Dev. Ctries. 2010, 15, 49–67. [Google Scholar]
  13. Le-Hoai, L.; Lee, Y.D.; Lee, J.Y. Delay and cost overruns in Vietnam large construction projects: A comparison with other selected countries. KSCE J. Civ. Eng. 2008, 12, 367–377. [Google Scholar] [CrossRef]
  14. James, D.O.; Lekan, M.A.; Oloke, C.O.; Olusanya, O.; Tunji-Olayeni, P.; Dele, P.O. Causes and effect of delay on project construction delivery time. Int. J. Educ. Res. 2014, 2, 197–208. [Google Scholar]
  15. Gamil, Y.; Rahman, I.A. Assessment of critical factors contributing to construction failure in Yemen. Int. J. Constr. Manag. 2020, 20, 429–436. [Google Scholar] [CrossRef]
  16. Annamalaisami, C.D.; Kuppuswamy, A. Reckoning construction cost overruns in building projects through methodological consequences. Int. J. Constr. Manag. 2022, 22, 1079–1089. [Google Scholar] [CrossRef]
  17. Bortolini, R.; Forcada, N. Building Inspection System for Evaluating the Technical Performance of Existing Buildings. J. Perform. Constr. Facil. 2018, 32, 04018073. [Google Scholar] [CrossRef]
  18. Lee, S.; Kim, B.; Noh, B. A Study on the Impact of Change Order Delay in the Apartment House Construction. J. Archit. Inst. Korea 2010, 26, 79–86. [Google Scholar]
  19. Yoon, S. A Study on the Solutions of Claims Due to Design Changes. Master’s Thesis, Kyung Hee University Graduate School, Seoul, Korea, 2010. [Google Scholar]
  20. Lee, J. Value Engineering for Defect Prevention on Building Façade. J. Constr. Eng. Manag. 2018, 144, 04018069. [Google Scholar] [CrossRef]
  21. Shoar, S.; Chileshe, N. Exploring the causes of design changes in building construction projects: An interpretive structural modeling approach. Sustainability 2021, 13, 9578. [Google Scholar] [CrossRef]
  22. Han, S.; Love, P.; Peña-Mora, F. A system dynamics model for assessing the impacts of design errors in construction projects. Math. Comput. Model. 2013, 57, 2044–2053. [Google Scholar] [CrossRef]
  23. Hossain, M.A.; Chua, D.K.H. Overlapping design and construction activities and an optimization approach to minimize rework. Int. J. Proj. Manag. 2014, 32, 983–994. [Google Scholar] [CrossRef]
  24. Rahman, I.A.; Ameri, A.E.S.; Memon, A.H.; Al-Emad, N.; Alhammadi, A.S.M. Structural Relationship of Causes and Effects of Construction Changes: Case of UAE Construction. Sustainability 2022, 14, 596. [Google Scholar] [CrossRef]
  25. Fan, C.L. Defect Risk Assessment Using a Hybrid Machine Learning Method. J. Constr. Eng. Manag. 2020, 146, 04020102. [Google Scholar] [CrossRef]
  26. Lin, C.L.; Fan, C.L. Examining Association between Construction Inspection Grades and Critical Defects using Data Mining and Fuzzy Logic. J. Civ. Eng. Manag. 2018, 24, 301–317. [Google Scholar] [CrossRef]
  27. Cho, S. Time Series Analysis Using Syntactic Pattern Based Quantitative Association Rule. Master’s Thesis, Hongik University Graduate School, Seoul, Korea, 2002. [Google Scholar]
  28. Shmueli, G.; Bruce, P.C.; Patel, N.R. Data Mining for Business Intelligence: Concepts, Techniques, and Applications with XLMINER, 3rd ed.; Wiley: Hoboken, NJ, USA, 2016; p. 509. [Google Scholar]
Figure 1. Research process.
Figure 1. Research process.
Applsci 12 11036 g001
Figure 2. Design of ARM model.
Figure 2. Design of ARM model.
Applsci 12 11036 g002
Table 1. Composition of original case data.
Table 1. Composition of original case data.
Source Data ConfigurationExplanation
Number of projects517
Year of project implementationCompleted within 2015–2021
Number of design changes35,194
Disciplines occured the design changesArchitectural, structural, mechanical, outdoor mechanical, electrical, communication, firefighting, earthwork, earthwork of district, building earthwork, landscape, and common (12 categories)
Reason for design changeDesign change, design change by head office, design change according to the request of government agencies, design errors, design improvement, site conditions, cost reduction, extension of schedule, change of finishing materials, reflecting permit requirements, project plan change, additional construction, cost saving, civil complaints, sales promotion, completion settlement, and others (17 factors)
Attributes defining design change itemsNumber, contract number, contract name, main contractor, number of contractors, design cost, expected cost, initial contract price, successful bid rate, contractor selection method, start date, contract signing date, announcement date, completion date, number of contract changes, number of design changes, increased cost by contract change, final contract price, final completion date, disciplines, reason for extension, increase/decrease cost by reason for design change, reason for design change, details, etc.
Table 2. Association rule analysis input data structure.
Table 2. Association rule analysis input data structure.
Data Group Structure for Analysis of Association Rules
Number of design change items6323
Data group structureTable format with 9 columns (number, region, main contractor, contractor selection method, contract price, disciplines, location/materials, reason for design change, detailed reason, etc.)
Number of projects299 projects
Disciplines11 disciplines (architecture, structure, mechanical, electrical, communication, firefighting, earthwork, landscaping, temporary work, outdoor mechanical, building earthwork, etc.)
Location/materials28 types (PIT floor, living room, stair room, common area, common facilities, common, machine room, etc., balcony, room, corridor, firefighting equipment, elevator, indoor piping, indoor wiring, rooftop, outdoor, outdoor piping, exterior wall, bathroom, electrical room, kitchen, main entrance, underground, underground water tank, underground parking lot, piloti, entrance, etc.)
Reason for design change10 reasons (extension of schedule, civil complaints, sales promotion, project plan change, design improvement, head office/government agency request, cost saving, completion settlement, site conditions, other, etc.)
Table 3. Association rule classification system through expert interview.
Table 3. Association rule classification system through expert interview.
DivisionItemSignExplanation
Strong rulesStrong rulesRules that are well-known and that anyone can easily predict.
New rulesRelevantan explainable ruleRules that are unknown until now, but seem relevant and explainable.
(Previously unknown but recently known information, etc.)
rules that are difficult to explainA rule that was not expected, but appears to be related by expert intuition and is difficult to explain.
Not relevantirrelevant rulesXUnexpected and unexplained rules.
Table 4. Results of derivation of association rules for all design change issue data.
Table 4. Results of derivation of association rules for all design change issue data.
No.LHS=>RHSSupportConfidenceLiftClassification
[1]{architecture, design change, underground parking lot}=>{architecture, design change, public facilities}0.1640.6812.098
[2]{earthwork, site conditions, etc.}=>{earthwork, site conditions, underground}0.1540.7302.695
[3]{architecture, design change, roof}=>{architecture, design change, public facilities}0.1540.6481.997X
[4]{architecture, design change, balcony}=>{architecture, design change, public facilities}0.1510.6341.954X
[5]{architecture, design change, underground parking lot}=>{architecture, design change, balcony)}0.1470.6112.574X
[6]{architecture, design change, balcony}=>{architecture, design change, roof}0.1400.5922.491
[7]{architecture, design change, public facilities}=>{architecture, design change, common}0.1400.4331.904
[8]{earthwork, site conditions, outdoor}=>{earthwork, site conditions, underground}0.1340.8513.142
[9]{architecture, design change, corridor}=>{architecture, design change, public facilities}0.1240.7122.193X
[10]{architecture, design change, underground parking lot}=>{architecture, design change, common}0.1200.5002.199
[11]{architecture, design change, underground parking lot}=>{architecture, design change, roof}0.1200.5002.106X
[12]{architecture, design change, public facilities}=>{architecture, design change, outdoor}0.1200.3712.094
[13]{architecture, design change, balcony}=>{architecture, design change, common}0.1170.4932.168
[14]{architecture, design change, corridor}=>{architecture, design change, balcony}0.1140.6542.754X
[15]{architecture, design change, outdoor}=>{architecture, design change, balcony}0.1100.6232.622X
[16]{architecture, design change, roof}=>{architecture, design change, outdoor}0.1070.4512.543X
[17]{architecture, design change, common}=>{architecture, design change, roof}0.1070.4711.982X
[18]{architecture, design change, outdoor}=>{architecture, design change, underground parking lot}0.1040.5852.429
[19]{architecture, site conditions, underground}=>{earthwork, site conditions, underground}0.1040.6462.384
[20]{architecture, site conditions, etc.}=>{earthwork, site conditions, underground}0.1040.5442.008
[21]{architecture, design change, roof}=>{earthwork, site conditions, underground}0.1040.4371.612X
[22]{architecture, design change, underground parking lot}=>{earthwork, site conditions, underground}0.1040.4311.589
●: strong rules, ◌: relevant and explainable rules, △: relevant rules but difficult to explain, X: irrelevant rules.
Table 5. Classification criteria for deriving rules for design change by project condition.
Table 5. Classification criteria for deriving rules for design change by project condition.
Project ConditionsCondition Classification/Number of Data
Contractor’s ranking1st to 30th place (1508)31st—Below 100th (2046)Over 100 (2769)
Contract priceLess than 50 billion won (4296)50~100 billion won (1175)Over 100 billion won (252)
Contractor selection methodQualification screening method (1257)Lowest bid method (4988)Turnkey method (78)
Table 6. Comparison of derivation result of design change association rule by project condition–contractor’s ranking.
Table 6. Comparison of derivation result of design change association rule by project condition–contractor’s ranking.
Contractor’s RankingAssociation Rule Analysis Result (Confidence Top 3 Rules)
1st–30th placeCharacteristic- Associated Rules Main Disciplines: architecture, machinery
- Main mechanism: Outdoor → Public facilities → Underground parking lot, kitchen → Entrance → Public facilities → Underground parking lot, etc.
- Main cause: Most of the design changes of public facility and unit space are caused by voluntary quality improvement activities to promote sales.
[1]{machinery, design change, public facilities}=>{architecture, design change, public facilities}
[2]{architecture, design change, balcony}=>{architecture, design change, underground parking lot}
[3]{architecture, sales promotion, entrance}=>{architecture, sales promotion, living room}
31st–100th placeCharacteristic- Associated Rules Main disciplines: earthwork, architecture
- Main mechanism: Outdoor piping → Outdoor → Corridor → Balcony, etc.
- Main cause: Failed to establish earthwork plan, resulting in site conditions and design changes → Estimation of changes in the design of unit households and common spaces in the type of discipline.
[1]{earthwork, site conditions, outdoor plumbing}=>{earthwork, site conditions, outdoor}
[2]{earthwork, site conditions, outdoor plumbing}=>{earthwork, site conditions, underground}
[3]{architecture, design change, outdoor}=>{architecture, design change, common}
Above 100Characteristic- Association Rules of Main Disciplines: architecture, earthwork, machinery
- Main mechanism: underground parking lot → public facilities → rooftop → outdoor, etc.
- Main causes: Changes to improve design errors in many disciplines, causing design changes in adjacent/connected spaces, and insufficient establishment of earthwork plans in the early stage of construction
[1]{earthwork, site conditions, outdoor}=>{earthwork, site conditions, underground}
[2]{earthwork, site conditions, etc.}=>{earthwork, site conditions, underground}
[3]{architecture, design change, stairwell}=>{architecture, design change, balcony}X
Table 7. Comparison of the results of deriving the rules related to the design changes by project conditions—contract price.
Table 7. Comparison of the results of deriving the rules related to the design changes by project conditions—contract price.
Contract PriceAssociation Rule Analysis Result (Confidence Top 3 Rules)
Under 50 billionCharacteristic- Associated Rules of Main Disciplines: architecture, earthwork
- Main mechanism: underground parking lot → public facilities → rooftop, common facilities → corridor → balcony → roof, etc.
- Main causes: Most items are changed due to design errors and quality control failure
[4]{earthwork, site conditions, etc.}=>{earthwork, site conditions, underground}
[9]{architecture, design change, corridor}=>{architecture, design change, balcony}X
[1]{architecture, design change, underground parking lot}=>{architecture, design change, public facilities}
500–under 100 billionCharacteristic- Association Rules of Main Disciplines: earthwork, construction, machinery
- Main mechanism: Rooftop → Underground parking lot → Balcony → Common facilities → Machine room → Underground
- Main causes: Insufficient design quality capability due to the nature of mid-scale construction, certain voluntary design changes to promote sales
[12]{machinery, design change, public facilities}=>{machine, design change, machine room}
[5]{earthwork, site conditions, outdoor}=>{earthwork, site conditions, underground}
[13]{architecture, sales promotion, kitchen}=>{architecture, design change, underground parking lot}
More than 100 billionCharacteristic- Association Rules of Main Disciplines: architecture, machinery, earthwork
- Main Mechanism: Room → Entrance/Living Room → Kitchen
- Main cause: Design change for all reasons, either in-house or owing to the head office request for sales promotion, unit generation quality improvement
[1]{architecture, sales promotion, room}=>{architecture, sales promotion, kitchen}
[2]{architecture, sales promotion, entrance}=>{architecture, sales promotion, kitchen}
[3]{architecture, sales promotion, living room}=>{architecture, sales promotion, kitchen}
Table 8. Comparison of the derivation results of the associated rules to design changes by project condition—contractor selection method.
Table 8. Comparison of the derivation results of the associated rules to design changes by project condition—contractor selection method.
Contractor Selection MethodAssociation Rule Analysis Result (Confidence Top 3 Rules)
Qualification screeningCharacteristic- Association Rules of Main Disciplines: Architecture
- Main mechanism: Underground parking lot → Public facilities → Balcony → Rooftop
- Main cause: Design change of underground parking lot construction type causes design change of other spaces (Limited to construction type, contractor Design change capability ↑)
[1]{architecture, design change, underground parking lot}=>{architecture, design change, public facilities}
[5]{architecture, design change, underground parking lot}=>{architecture, design change, balcony}
[4]{architecture, design change, balcony}=>{architecture, design change, roof}
Lowest priceCharacteristic- Association Rules of Main Disciplines: earthwork, architecture
- Main mechanism: underground parking lot → rooftop → outdoor → underground parking lot, etc.
- Main cause: Failure to change the design of common parts (corridor, underground parking lot, rooftop, outdoor, etc.) induce other space design changes
[5]{earthwork, site conditions, outdoor}=>{earthwork, site conditions, underground}
[1]{earthwork, site conditions, etc.}=>{earthwork, site conditions, underground}
[13]{architecture, design change, outdoor}=>{architecture, design change, public facilities}
TurnkeyCharacteristic- Associated Rules of Main Disciplines: electricity, machinery, architecture
- Main Mechanism: Indoor Plumbing → Balcony → Entrance/Kitchen/Living Room
- Main cause: A large number of design changes for indoor piping due to the design change request of the head office → Changes in the design of major spaces in unit households to promote sales
[1]{electricity, sales promotion, balcony}=>{Machine, design change (headquarters), indoor piping}
[2]{electricity, sales promotion, balcony}=>{machine, sales promotion, balcony}
[3]{Machine, design change (headquarters), indoor piping}=>{machine, sales promotion, balcony}
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Kim, M.; Lee, J.; Kim, J. Analysis of Design Change Mechanism in Apartment Housing Projects Using Association Rule Mining (ARM) Model. Appl. Sci. 2022, 12, 11036. https://doi.org/10.3390/app122111036

AMA Style

Kim M, Lee J, Kim J. Analysis of Design Change Mechanism in Apartment Housing Projects Using Association Rule Mining (ARM) Model. Applied Sciences. 2022; 12(21):11036. https://doi.org/10.3390/app122111036

Chicago/Turabian Style

Kim, Moonhwan, Joosung Lee, and Jaejun Kim. 2022. "Analysis of Design Change Mechanism in Apartment Housing Projects Using Association Rule Mining (ARM) Model" Applied Sciences 12, no. 21: 11036. https://doi.org/10.3390/app122111036

APA Style

Kim, M., Lee, J., & Kim, J. (2022). Analysis of Design Change Mechanism in Apartment Housing Projects Using Association Rule Mining (ARM) Model. Applied Sciences, 12(21), 11036. https://doi.org/10.3390/app122111036

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop