Next Article in Journal
Optimized Pilot Point Emplacement Based Groundwater Flow Calibration Method for Heterogeneous Small-Scale Area
Next Article in Special Issue
State-of-the-Art in Orthodontics and Gnathology
Previous Article in Journal
Induction Machine-Based EV Vector Control Model Using Mamdani Fuzzy Logic Controller
Previous Article in Special Issue
A 3D CBCT Analysis of Airway and Cephalometric Values in Patients Diagnosed with Juvenile Idiopathic Arthritis Compared to a Control Group
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Applied Sciences
Volume 12
Issue 9
10.3390/app12094646
applsci-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Dento-Skeletal Class III Treatment with Mixed Anchored Palatal Expander: A Systematic Review

by
Fabrizia d’Apuzzo
1,†,
Ludovica Nucci
1,*,†,
Bruno M. Strangio
1,
Alessio Danilo Inchingolo
2,*,
Gianna Dipalma
2,*,
Giuseppe Minervini
1,
Letizia Perillo
1,‡ and
Vincenzo Grassia
1,‡
1
Multidisciplinary Department of Medical-Surgical and Dental Specialties, University of Campania Luigi Vanvitelli, 80138 Naples, Italy
2
Interdisciplinary Department of Medicine, University of Bari Aldo Moro, 70124 Bari, Italy
*
Authors to whom correspondence should be addressed.
†
These authors contributed equally to this work.
‡
These authors contributed equally to this work.
Appl. Sci. 2022, 12(9), 4646; https://doi.org/10.3390/app12094646
Submission received: 15 February 2022 / Revised: 28 April 2022 / Accepted: 29 April 2022 / Published: 5 May 2022
(This article belongs to the Special Issue State-of-the-Art in Orthodontics and Gnathology)
Browse Figures
Versions Notes

Abstract

:
Bone-anchored appliances for the treatment of Class III malocclusions have recently been found to reduce the dentoalveolar effects caused by conventional tooth-borne devices while also improving orthopaedic outcomes in growing patients. The goal of this systematic review was to compare the outcomes of skeletal Class III interceptive treatment with dental anchoring devices to those achieved with mixed anchored palatal expanders. The inclusion criteria were as follows: patients who were treated with hybrid anchored palatal expanders and different maxillary advancement appliances. Study quality was estimated using the Newcastle–Ottawa scale. A search on the Pubmed, Scopus, Embase and Cochrane Library databases yielded 350 papers. Following the initial abstract selection, 65 potentially acceptable papers were thoroughly examined, resulting in a final selection of 9 articles. The results in the short-term showed that combined tooth-borne and bone-borne appliances for rapid maxillary expansion might be recommended in protocols of skeletal Class III treatment to obtain more skeletal effects and reduce side effects on the upper teeth.

1. Introduction

The treatment of skeletal Class III malocclusion is sometimes a challenge in orthodontics. The prevalence of this type of malocclusion presents high variability among and within populations ranging from 0% to 26%: the populations from Southeast Asian countries (Chinese and Malaysian) showed the highest prevalence rate of 15.8%, Middle Eastern nations had a mean prevalence rate of 10.2%, European countries had a lower prevalence rate of 4.9% and the Indian population showed the lowest one of 1.2% [1]. The etiology of Class III malocclusion is generally genetic, as has been demonstrated in several studies. [1,2,3]. A wide range of environmental factors have been suggested as contributing to the onset of Class III malocclusion (enlarged tonsil, difficulty nasal breathing, habit of protruding the mandible) [2,3,4,5,6]. Craniofacial features may be attributed to the incongruity of position and the size of the craniofacial structures at the skull base, maxilla and/or mandible [7,8,9,10,11,12,13,14,15]. Several tooth-borne anchorage treatments have been proposed to treat Class III dentoskeletal disharmony, including intraoral and extraoral appliances, such as the rapid maxillary expansion [16,17,18,19] along with the facial mask (RME/FM) and two occlusal acrylic splints associated with Class III elastics and chin-cup (SEC III) [20,21].
Some adverse effects were reported with the use of conventional dental anchorage as RME/FM such as upper incisors proclination and extrusion and mesial tipping of the upper molars, gingival recessions, [3,22,23,24,25,26,27,28] fenestrations of the buccal cortex and root resorption of the posterior teeth [29,30,31,32,33,34]. The SEC III appliances also present some limits, such as uncomfortable dimensions of splints and the impossibility of having expansion in the upper arch. Therefore, a modified SEC III protocol including a maxillary bonded expander has been used with the remaining limit of the mesializing effect of the upper arch [20]. To overcome tooth-borne anchorage treatment limitations, the use of bone anchorage has recently been proposed [10,35,36,37,38]. The use of micro-implants allows for the achievement of skeletal anchorage without the need for surgical procedures such as mini-plate placement and removal [39,40,41,42,43,44,45,46]. The goal of this systematic review of the literature is to determine the efficacy of using a mixed anchored palatal expander to treat Class III malocclusions, as well as to see if using a bone-anchorage device induces more maxillary advancement with fewer dental side effects.

2. Materials and Methods

The authors registered this systematic review on PROSPERO, the International Prospective Register of Systematic Reviews (Centre for Reviews and Dissemination, University of York, York, UK). The protocol was under registration at PROSPERO with the number CRD42022207212.

2.1. Search Strategy

The bibliography was rigorously evaluated in accordance with PRISMA 2020 guidelines (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) [17]. Pubmed, Scopus, Embase and Cochrane Library databases were extensively used for research, with no limit in terms of publication date, in September 2021. The keywords include Mesh and non-Mesh terms to limit the field of study. The research strategy was: “skeletal anchorage or bone anchor or miniscrew or mini-implant or bone screw” combined with “skeletal Class III or mandibular prognathism or mandibular hyperplasia or maxillary retrusion or maxillary hypoplasia or mandibular protrusion or Angle Class III” and “interceptive treatment or early treatment or orthopedic treatment or interceptive orthodontic or interceptive or early therapy”. These terms were combined in different ways, and further studies cited in the included articles were analyzed.
Title and abstract screening were performed to select articles for full-text retrieval by two reviewers (B.M.S. and L.N.). To find potentially relevant papers, an initial screening of titles and abstracts against the inclusion criteria was undertaken, followed by a review of the complete potentially relevant papers. Duplicate publications were deleted, and studies were chosen for inclusion by both authors separately. The two reviewers had a concordance rate of less than 3%, and any doubts or disputes were resolved following conversation.

2.2. Selection Criteria for the Studies Included in This Review

The inclusion criteria were the following: published articles, articles in press and reviews concerning studies in humans. Randomized clinical trials (RCTs), case–control studies and retrospective and prospective cohort studies were accepted. The exclusion criteria were: case reports, case series, literature reviews, systematic reviews, meta-analyses and editorials and any articles including animal or laboratories studies or patients with syndromes or craniofacial deformities or who had undergone maxillofacial surgery. The eligibility sample criteria were growing patients with skeletal Class III malocclusion who had undergone orthodontic treatment with mixed-anchored palatal expanders.

2.3. Data Collection Process

The variables recorded for each article reviewed were: author, aim, sample size, demographic variables (gender, age), treatment used and the study results. A customized data collection form was created to gather information from the selected studies.

2.4. Types of Outcomes

The primary outcome was the evaluation of skeletal changes after Class III treatment using a mixed anchored palatal expander. The secondary outcome was to compare results obtained using this protocol and others with tooth-borne anchorage.

2.5. Quality Assessment

The Newcastle–Ottawa scale was used by the same researchers to assess the quality of the studies [47]. In case of a disagreement between the two initial researchers, a consensus was reached, and the third researcher was consulted in case of question.

2.6. Risk of Bias Assessment

The evaluation of the risk of bias for the selected studies was carried out independently by B.M.S. and L.N., using the Cochrane Collaboration tool (Figure 1). In case of disagreement, the third author (V.G.) was consulted. A consensus was reached through discussion. Risk of bias rated as “low,” “high” or “unclear” included the following: random sequence generation, sample size, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, selective reporting and other biases.

3. Results

3.1. Study Selection and Flow Diagram

The initial search identified a total of 350 articles. After removing 73 duplicates, 277 articles were screened and 226 articles were excluded after reading the title and abstract due to their poor relevance to the research question. The remaining 65 articles were analyzed. Among these, 56 were excluded for these reasons: 13 were case reports, 12 were case series, 1 was an animal study, 9 included adult patients, 5 showed orthognathic surgery, 9 did not use palatal screws and 7 was an in vitro study. At the end, 9 articles were included in the qualitative synthesis (Figure 2).

3.2. Study Characteristics

On the Newcastle–Ottawa scale, the 9 studies included in this study had varying levels of quality [25] as shown in Table 1. A total of 5 studies [4,15,29,48,49,50] presented low to moderate quality, whereas 4 [1,51,52,53] presented high quality.
Of the 9 studies, one was a randomized clinical trial, 5 were case-control studies and 3 were cohort studies. In 6 studies, a bone-anchored palatal device group was compared with the control one. One of these did not receive any treatment. Another was treated with combined tooth and bone-borne appliances [50]. In two studies, the control groups were treated with tooth-borne appliances and facemasks [52,54], and in the last one, the control group was treated with hybrid Hyrax and mentoplates [53].
The patients’ characteristics in the selected studies were: growing phase; skeletal and molar Class III malocclusion; anterior crossbite and/or edge-to-edge occlusion.

3.3. Qualitative Synthesis of the Studies Included

The qualitative analyses of the studies included were performed (Table 2). Al-Mozany et al. [4] and Almuzian et al. [15] selected 14 participants (7 M and 7 F; average age: 12.05 ± 1.09 years) with these features: Cervical Vertebral Maturational (CVM) Stage 2 or 3; retrognathic or hypoplastic maxilla; anterior crossbite and dental Class III molars and canines. The exclusion criteria were: previous orthodontic/orthopedic treatment or congenital abnormalities. All records (T1) were taken in the centric relation before starting the intervention.
Maino et al. [49] selected 28 patients (15 males and 13 females; mean age: 11.4 ± 2.5 years). The inclusion criteria were: growing phase; Class III malocclusion (evaluating Wits index). The exclusion criteria were craniofacial syndromes and previous orthopedic or orthodontic treatment. Wilmes et al. [29] selected 13 patients (7 females and 6 males; mean age: 11.2 years), but they did not report the inclusion and exclusion criteria. Nienkemper et al. [51] selected a treatment group of 16 patients and a control group of 16 untreated patients. At the start (T1), all patients showed mixed dentition, with a Wits index of –2 mm or less (mean, −5.6 ± 2.2 mm), anterior crossbite or incisor edge-to-edge relationship and a Class III molar relationship. According to the Cervical Vertebral Maturation method, all patients were in the prepubertal stage of skeletal maturity (CS1-3). Hourfar et al. [50] selected 100 patients (59 females and 41 males; mean age: 13.04 ± 4.82 years) with transverse deficits of the maxilla; all had been treated by RME without surgical support in the context of orthodontic treatment indications. A total of 2 groups of 50 patients were formed, including a conventional group of 29 females/21 males treated with strictly tooth-borne appliances and a hybrid group of 30 females/20 males treated with appliances anchored in both the teeth and jawbone. The patients were further divided into the skeletal Class I (0° < ANB ≤ 4°) or Class III (ANB ≤ 0°) subgroup based on their pretreatment cephalometric findings. The other inclusion criteria were Caucasian descent and bilateral posterior crossbite. The exclusion criteria were: previous orthodontic treatment, the extraction of permanent teeth, planned extractions, congenital agenesis of permanent teeth, craniofacial anomalies or trauma, systemic disease, trauma of frontal teeth with or without tooth loss, or maxillary protraction (facemask). Seiryu et al. [55] selected 39 patients, 20 treated only with facemasks (FM group) and 19 treated with facemasks and miniscrews (FM + MS group). Ngan et al. [54] selected 40 Class III patients, 20 of whom received the tooth-borne maxillary RPE and protraction device and 20 of whom received a bone-anchored maxillary RPE and protraction appliances. Willmann et al. [53] selected 34 patients for hybrid hyrax appliance treatment, 17 for the facemask (mean age 8.74 ± 1.20) and the other 17 for mentoplate (mean age 9.43 ± 0.95) for maxillary advancement.

3.4. Skeletal Anchorage

Various methods have been used, which excluded any skeletal anchorage: Maino et al. [49] treated patients using a rapid palatal expander (RPE) with hybrid anchorage. Two paramedial palatal miniscrews were inserted; the RPE’s anterior metal arms were welded to two metal abutments that fit over the miniscrews’ heads. Al-Mozany et al. [4] and Almuzian et al. [15] inserted two paramedial palatal miniscrews and two mandibular miniscrews between the canine and lateral incisors. The Hybrid MARME was cemented with a glass ionomer cement on day 28 of the miniscrew insertion. The MLA (modified lingual arch) was built and cemented. The lingual cleats that protruded from the MLA were attached with composite resin to the lingual surfaces of the anterior teeth. Wilmes et al. [29] applied two mini-implants in the anterior palate next to the midpalatal suture and near the second and third palatal rugae. Then, 7 to 10 days after placing the mini-implants, the hybrid hyrax appliance was inserted and connected to the miniscrews and the first permanent molars. Nienkemper et al. [51] placed two mini-implants on opposite sides of the midpalatal suture in the anterior part. A split palatal screw, two orthodontic bands attached to the first molars and two abutments screwed to mini-implants made up the hybrid hyrax appliance. Rigid stainless-steel wire with a diameter of 1.5 mm was used to connect these components. Rigid sectional wires were soldered to the buccal side of the molar bands to apply orthopedic protraction forces. Hourfar et al. [50] used two miniscrews in the anterior palate at paramedian locations. These miniscrews supported the anterior side of a hybrid hyrax, while the posterior side was connected to the orthodontic bands on the first molars, with RME force applied via a centrally located expansion screw. Seiryu et al. [55] inserted a miniscrew in the anterior region of the palate. Ngan et al. [54] placed two mini-implants in the area of the third palatal rugae. Willman et al. [53] placed two paramedian screws in the anterior palate to fit a Hybrid-Hyrax, while in patients treated with mentoplates, the device was inserted in the subapical region of the lower incisors.

3.5. Expansion Activation Protocol

Maino et al. [49] used a RME with hybrid anchorage according to the Alt-Ramec protocol (alternation of expansion and compression of the maxillary complex). Al-Mozany et al. [4] and Almuzian et al. [15] also used the Alt-Ramec protocol. Wilmes et al. [29] activated the sagittal split screw twice a day by a 90° turn immediately after insertion of the hybrid hyrax. This resulted in a daily activation of 0.8 mm. RPE was continued until a 30% overcorrection was achieved. The split screw was activated by 90° turns 4 times/day by Nienkemper et al. [51], resulting in an expansion of 0.8 mm/day. The activation was continued until a 30% transverse overcorrection was obtained. Hourfar et al. [50] activated the expansion screw of the RME appliance 3 times/day for a change of 0.2 mm per activation. Seiryu et al. [55] used a screw to support a lingual arch, and no expansion device was used. Ngan et al. [54] instructed patients to activate the jackscrew twice daily for one week, whereas patients with constricted maxilla activated for two weeks. Willman et al. [53] performed the RME activating the screw four times a day.

3.6. Class III Biomechanics

Maino et al. [49]: maxillary protraction was achieved via facemask, which was to be worn 14 h per day for 4 months. The protraction elastics (400 g per side) were attached near the maxillary canines, with a downward and forward pull of 30° from the occlusal plane. Al-Mozany et al. [4], Almuzian et al. [15]: prescription was written for two full-time heavy intra-oral elastics per side, totaling 400 g per side One of these elastics ran in the long-closing Class III configuration, from the posterior ball clasps on the hybrid MARPE to the “S” hook. The other ran from the anterior hook on the hybrid MARPE to the MLA in a short-closing Class III configuration. In Wilmes et al. [26] a facemask was prescribed for approximately 6 months to simultaneously protract the maxilla. The applied elastics (142 g) were anterocaudally angulated. Facemasks were used by Nienkemper et al. [51] to achieve maxillary protraction. The elastics were applied with an inclination in both directions (downward and forward) of 20–30° from the occlusal plane. The elastics delivered 400 g of force per side, which was controlled by a force gauge. The patients were told to wear the facemask for 16 h/day. Hourfar et al. [50] used only a RPE, with no sagittal force directed on the maxilla. Seiryu et al. [55] applied a protractive force of 250 g per side from the facemask to the hooks using elastics. The FM has been worn for 12 h per day with a direction of traction force < 3° from the occlusal plane. Ngan et al. [54] attached heavy elastics to generate the maxillary advancement, which delivered 380 g per side. The prescription was FM for 12–14 h a day. Willman et al. [53] instructed the facemask patients to wear 400 g elastics per side for 14–16 h per day, adjusting the force vector to have an inclination of 20–30° to the occlusal plane.

3.7. Dentoalveolar Effects

Al-Mozany et al. [4] and Almuzian et al. [15] found a significant proclination of the maxillary incisors (UI-SN +2.98 ± 2.71°, p < 0.01) and a retroclination of the mandibular incisors (LI-MP −3.2 ± 3.4°, p < 0.05). Maino et al. [49] documented that the maxillary molar had slight extrusion (U6 vert PP 0.42 mm) and mesialization (U6 mesialization 0.87 mm). The average forward displacement of the incisors was 3.62 mm (Pr-VertT), and their retroclination was 2.26° with respect to the palatal plane (U1-PP), according to Maino et al. [49]. Wilmes et al. [29] documented only a mesial migration of the maxillary incisors. The right first molar migrated 0.4 ± 0.6 mm mesially and the left one 0.3 ± 0.2 mm. In addition, Al-Mozany et al. and Almuzian et al. [4,15] recorded a significant improvement in the overjet: 5.63 ± 1.36 mm (p < 0.001). Both the manuscripts of Maino and Wilmes [29,49] did not report the values of OVJ and OVB. Nienkemper et al. [51] recorded a significant improvement in the OVJ and molar relationship (3.2 mm and –3.1 mm, respectively) in the treatment group, but no significant differences between the treatment group and control group (untreated Class III subjects) could be found in OVB (−0.2 mm) or inclination of the maxillary (U1-PP: −0.5°) and mandibular incisors (L1-MP: −1.7°). Hourfar et al. [50] did not analyze the dentoalveolar effects of the hybrid hyrax but only the skeletal effects by pre- and post-treatment lateral cephalograms of the patients. Seiryu et al. [55] referred a major proclination of the maxillary incisors in the FM group in comparison to the FM + MS group (FM group: 4.6 ± 4.5°; FM + MS group: −0.4 ± 4.2°; p < 0.01), due to the greater improvement of the skeletal relationship in patients treated with palatal anchorage. Ngan et al. [54] found significant and greater change in OVJ in the tooth-borne group (5.5) compared to the bone-anchored group (3.4 mm, p < 0.001). This is due to the tooth-borne group having more forward migration of the maxillary incisors (OLp-Is 2.12 vs. 0.87 mm, p < 0.05). Maxillary incisors presented a significantly greater downward movement and palatal inclination in the bone-anchored group (Is-NL 1.34 mm, Is-SNL −4.42°) compared to the tooth-borne group (−0.55 mm, −0.19°, p < 0.05). Furthermore, the tooth-borne protraction facemask group had a significant and bigger change in molar relationship (2.7 mm) than the bone-anchored group (1.1 mm, p < 0.05), which was due to a greater differential movement of maxillary and mandibular molars in the tooth-borne group. Willman et al. [53] found no statistically significant difference in upper incisor proclination, space loss for the canines and mesial migration of the molars between the study groups. The majority of the overjet correction (FM group 3.51 mm, ME group 3.06 mm) was attributable to skeletal effects rather than dentoalveolar compensation.

3.8. Skeletal Effects

In sagittal dimension, Al-Mozany et al., Almuzian et al. and Dekel et al. [4,15,56] showed maxillary protraction (SNA 1.87 ± 1.06°; Vert.T-A 3.29 ± 1.54 mm p < 0.001), whereas the mandibular base significantly retropositioned (SNB −2.03° ± 0.85°; Vert.T-B −3.43 ± 4.47 mm, p < 0.001 and p < 0.05, respectively), resulting in a better skeletal relationship (ANB 3.95° ± 0.57°, p < 0.001; Wits 5.15 ± 1.51 mm, p < 0.001). Maino et al. [49] also showed a maxillary advancement (SNA +2.50°; Vert.T-A +3.4 mm), while the mandibular base redirected posteriorly (SNB −0.92°; Vert.T-B −0.26 mm). The mandibular plane rotated clockwise, improving the ANB (+ 3.41°) and the Wits index (+ 4.92 mm). Wilmes et al. [29] did not consider these parameters but specified only an improvement of Wits appraisal (2.7 ± 1.3 mm) and an expansion in the first premolar/first primary molar region and in the first molar region (+ 6.3 ± 2.9 mm and +5.0 ±1.5 mm, respectively). Nienkemper et al. [51] recorded an increase of 2.4° and 2.4 mm SNA and A point to Nasion perpendicular, respectively. Moreover, a significant reduction in CoGn (−2.3 mm) and SNB (−1.7°) was recorded. The Wits index increased by 4.5 mm, and the ANB angle improved by 4.1°. Hourfar et al. [50] recorded an SNA increase of 2°, and the maxilla underwent statistically significant amounts of caudal movement (S-Spa: +3.20; S-Spp: +1.21; N-Spa: +1.84; N-Spp: +2.89), whereas the maxillary inclination had not changed (NL/NSL: −0.06°). Furthermore, a mean mandibular post-rotation was assessed (ML/NSL: +1.46).
For the vertical measurements, Maino et al. [49] recorded that the facial angle (SN-GoGn) increased by 1.64° during treatment and the SN-PP angle was reduced by 1.11°. Al-Mozany et al. and Almuzian et al. [4,15] indicated Y-axis and lower third (ANS-Me) significant increases (1.95 ± 1.22° and 3.19 ± 2.2 mm, respectively), indicating a post-rotation of the mandible. The middle facial height (N-ANS) (0.32 ± 1.53 mm) showed no significant increase. Nienkemper et al. [51] reported that vertical growth was contained as indicated by a slight increase in the FMA angle (0.5°) and a little reduction in the Co-Go-Me angle (2.0°). The other skeletal vertical values showed no significant differences. Hourfar et al. [50] revealed increases in the vertical parameters ML/NSL (+1.46) and Björk sum (+1.46). Seiryu et al. [55] reported that both FM and FM + MS patients showed a significant increase in maxillary forward growth without mandibular forward growth. During the active treatment, SNA (FM + MS group: 2.28 ± 1.38; FM group: 1.18 ± 1.08; p < 0.01), SN-ANS (FM + MS group: 2.58 ± 1.78; FM group:1.28 ± 1.38; p < 0.05) and ANB (FM + MS group: 2.08 ± 1.38; FM group: 1.28 ± 1.28; p < 0.05) change was significantly greater in the FM + MS group than in the FM group.
Ngan et al. [54] found similar maxillary advancement (OLp-A pt., 0.7 mm) and mandible retropositioning (OLp-Pg, 2.2 mm) in both groups. Both the tooth-borne (Wits 2.19 mm, ANB 2.58°) and the bone-anchored (Wits 2.31 mm, ANB 2.17°) groups improved their anteroposterior jaw relationship. Other than that, the tooth-borne (OLparallel–A pt. 1.2 mm) protraction facemask group had a substantially larger downward displacement of the maxilla than the bone-anchored protraction facemask group (−0.4 mm, p < 0.005). The tooth-borne group had a considerably higher mandibular plane angle (SNL–ML 2.76°) than the bone-anchored protraction face-mask group (−0.25°, 0.23°, p < 0.05). Willman et al. [53] reported a significant improvement in the maxillary position in both groups. Similar changes in the SNA-Angle (SNA + 2.23°) and WITS-appraisal (FM Group 4.81 mm, ME 4.14 mm) during a comparable treatment period were found, while an SNB angle decrease difference was found in the study groups. In the FM-group, the skeletal effect on the mandible is more vertical, as evidenced by a posterior rotation of the mandible and a significant opening of the interbase angle (ML-NL). In the ME-group, however, the B-point remained stable, while the gonial angle decreased significantly, possibly due to changes in condylar and ramus growth (Table 3).

3.9. Soft Tissue Analysis

In Al-Mozany and Almuzian’s [15,57] study, cephalometric analysis of soft tissues revealed a significant increase in the H angle, at 2.76° ± 1.8° (p < 0.001). None of the remaining authors [26,46,47,48,49,50,51] reported soft tissue effects.

3.10. Quality Assessment

Al-Mozany et al., Almuzian et al. and Maino et al. [15,49,57] did not have a control group, and the patients were not randomized. Wilmes et al. [29] was a randomized study with a control group of 10 subjects. Nienkemper et al. [51] was a controlled clinical study with a control group of 16 subjects. It was not a randomized study. Hourfar et al. [50] was a retrospective cephalometric study with a control group of 50 subjects. It was not a randomized study. Al-Mozany et al. and Almuzian et al. [15,57] specified that no sample size calculation was undertaken. Maino et al., Wilmes et al. and Hourfar et al. [29,49,50] did not report a power analysis. Nienkemper et al. [58] specified that prior pilot research was used to calculate the sample size calculation. Based on a significant increase in SNA of 2.0° with a σ of 1.9°, an α level of 0.05 and a power of 0.80, the required sample size was found to be 16 patients in the treatment and control groups, respectively. Al-Mozany et al. and Almuzian et al. [15,57], in the section of the statistical analysis, reported that a paired-sample t-test (p < 0.05) was used to compare each variable from T1 to T2, and an error measurement (Dahlberg’s formula) study was conducted to evaluate the intra-examiner reliability, while, Maino et al. [49] used the Student t-test to check whether the pretreatment and post-treatment variations were significant (p < 0.05). Wilmes et al. [29] did not report a statistical analysis. According to Nienkemper et al., the data did not demonstrate normal distribution in an exploratory analysis using the Shapiro–Wilk test [58]. As a result, nonparametric statistics were applied. The Mann-Whitney U-test was used to look for significant differences between the cephalometric variables in the treatment and control groups at T1 (comparison of beginning forms) and during the T1-T2 interval. Hourfar et al. [50] performed paired t-tests for the intragroup and analysis of variance (ANOVA) for the intergroup comparisons. At p < 0.05, the results were considered statistically significant.
The research study design used by Seiryu [55] was a single-center, prospective randomized clinical trial. The sample size was calculated using data from a previous study that compared the treatment effects of a combination of bone-anchored maxillary protraction (BAMP) and facemask with a rapid maxillary expander (RME/FM) for maxillary advancement. With a test power of 80%, a significance level of 5% and an effect size of 0.98, the authors reported that treatment with facemask therapy and miniscrews resulted in 1.5 times increase in maxillary forward growth. For each group, a sample size of 18 patients was recommended. A computer-generated 1:1 randomization was carried out by someone who was not involved in the study. To check for normal distribution, all values were subjected to the Shapiro–Wilk test. Welch’s t-test was used to assess the significance of differences in all values that showed normal distribution (age, treatment period, cephalometric variables), while the Mann-Whitney U-tests were used to assess those that did not. * p < 0.05, ** p < 0.01 were used to indicate statistical significance.
Ngan et al. [54] used a two-tailed t-test with a confidence level of 95% to compare the starting forms of the control and experimental samples, as well as the skeletal and dental alterations between the groups at the two time periods. The reliability of cephalometric readings was determined using the intraclass correlation coefficient of reliability (R). All cephalometric variables had correlations ranging from 0.96 to 0.99, with the majority exceeding 0.98 (R value greater than 0.90 indicating high reliability). Willman et al. [53] used the Shapiro–Wilk test to examine the normal distribution of the measurements. Student’s t-test for dependent samples or the Wilcoxon test were used to find intra-group differences. The Mann–Whitney U test or the t-test for independent samples were used to test the differences between the groups. The 95% confidence of interval was chosen.

4. Discussion

The purpose of this systematic review of the literature was to determine the effectiveness of treating Class III malocclusions using a mixed anchored palatal expander [56]. Due to a scarcity of studies on this issue, only nine publications, largely case series, were chosen for the following work (characterized by a high research specificity and innovative protocol aspect).
All of the studies that were analyzed include in their protocol an RME appliance applied on a mini-screw anteriorly in the palate to obtain skeletal anchorage. Since the first molars are also included in the device design, it can be denominated as a bone- and tooth-borne appliance (hybrid hyrax). Al-Mozany et al., Almuzian et al. and Maino et al. [4,15,49] employed the Alt-Ramec protocol, which resulted in a more significant disarticulation of the two parts of the maxilla than the other methods of maxillary expansion examined in this research.
Because the ability of sutures to respond to therapy decreases with age, it has been suggested that starting maxillary traction with a facemask during the early mixed dentition period (around 8 years old) will provide the most skeletal benefit [47]. Skeletal anchorage, on the other hand, is treated at a later age, around 10 years old, when the characteristics of the bone allow for easier placement and stability [57]. When compared to pure bone-borne RPE devices such as distractors, the hybrid hyrax is surgically minimally invasive, and this is also important to avoid any periodontal inflammation [58,59,60,61]. The use of first molars and mini-screws as, respectively, anterior and posterior anchoring units has various advantages. In fact, according to Wilmes et al. [29], the hybrid hyrax can be employed even in individuals who have decreased anterior dental anchoring owing to missing primary molars, primary molars with resorbed roots or underdeveloped premolar roots. Moreover, the anterior teeth are excluded in the appliance, and regular orthodontic treatment can, therefore, be started early. Furthermore, in all tests, the miniscrews were implanted at paramedian sites in the anterior palate. This configuration is particularly beneficial for bone availability and safe isolation from vascular and nerve connections. The optimal size and direction of miniscrew insertions are identified on a cone-bean computed tomography (CBCT) scan and a 3-dimensional surgical guide to provide safe and reliable palatal miniscrew insertion [49].
Another significant benefit is that the combination of the hybrid hyrax with a maxillary protraction facemask is helpful to minimize dental adverse effects, such as teeth mesial migration. Al-Mozany et al. and Almuzian et al. [4,15] specify that the teeth were partially implicated in the maxillary protraction utilizing hybrid hyrax for the transfer of forces to the underlying skeletal systems. The frequent unfavorable consequences of tooth-anchorage devices, such as buccal proclination of the upper incisors and lingual inclination of the lowers, are eliminated with this procedure. As a result, the therapy had both skeletal and dentoalveolar effects (maxilla protraction and mandible posterior displacement).
Furthermore, the upper incisors displayed a forward displacement of 3.62 mm (Pr-VertT) and a retroclination of 2.26 degrees in reference to the palatal plane (U1-PP), according to Maino et al. [49]. It is likely that the retroclination found in many patients treated with a bone-anchored facemask or hybrid tooth-skeletal anchoring is due to the treatment’s skeletal benefits and the resultant diminution of the no-longer-required dentoalveolar compensation. However, despite the anchorage provided by the two mini-screws, Maino et al. [49] assessed the forward movement of the maxillary molars (although by less than 1 mm in all cases). The hybrid hyrax, according to Wilmes et al. [29], minimized excessive forward movement of the upper molars produced by facemask protraction (1.6 migrated 0.4 ± 0.6 mm mesially and 2.6 0.3 ± 0.2 mm). Nienkemper validated these findings, recording a modest molar mesial migration (approximately 0.4 mm) [51].
In the Nienkemper et al. [55], Al-Mozany et al. and Almuzian et al. [15,51] studies, the OVJ was improved. Nienkemper et al. confirmed that the OVB had not improved significantly, although Al-Mozany et al. claimed an improvement in the results section, but this did not match the data in the final table. Because Maino et al., Wilmes et al. and Hourfar et al. [29,49,50] did not show the OVB and OVJ values, it is difficult to know for sure how the OVB and the OVJ changed after therapy with a hybrid palatal expander. The rise in the angle SNA and the Wits assessment revealed a considerable skeletal improvement at the maxillary in the sagittal plane. These values are higher than those obtained with dental anchorage, indicating that skeletal anchorage is superior to dental anchorage because the exerted force was supported by a skeletal component as well as by teeth.
Furthermore, all studies indicated a posterior mandibular rotation using their approach (except Wilmes et al. [29], who did not evaluate the location of the jaw).
These combined, which include maxillary advancement, maxillary caudal movement and a posteriorly shifted mandibular base, reflect a pattern of three-dimensional geometric change that, according to Hourfar et al. [50], has the potential to facilitate the treatment of skeletal Class III patients, but the mandibular clockwise rotation is also a compensation for the treatment of Class III malocclusions [50]. Finally, patients who had mixed anchorage with miniscrews had higher maxillary protraction, which reduced unfavorable dental consequences. The fact that the orthopedic force acts directly on the surrounding sutures, increases the skeletal impact and eliminates dental compensation is one probable explanation [52]. However, these results will need to be verified with more randomized clinical trials and long-term follow-up. According to Seiryu et al. [55], the FM + MS group’s proclination of the maxillary anterior teeth at T1 was reduced at T2 by improving the maxilla–mandibular relationship. When compared to orthopedic force alone, the use of miniscrews [52] had fewer negative side effects on the maxillary teeth. As a result, whereas mandibular growth was driven forward and lower, neither clockwise rotation nor posterior displacement of the mandible occurred, showing that face-mask treatment with a miniscrew caused less posterior displacement than miniplates and elastics [55].
Ngan et al. [54] and Willman et al. [53] found that both the tooth-borne and bone-borne groups had identical maxillary protraction; however, Seiryu et al. [55] found that the maxillary protraction in the miniscrew group was bigger by twofold. This discrepancy could be due to the appliance utilized, the treatment length or the age at which the treatment began.
The tooth-born protraction facemask group had more forward movement of the maxillary than the bone-anchored protraction facemask group, resulting in a bigger rise in the OVJ, according to Ngan et al. [54]. Despite the anchorage, the maxillary molars migrated forward by an average of 0.6 mm in the bone-anchored groups; this was most likely due to wire bending rather than mini-implant movement. By combining retroposition of the mandible with advancement of the maxilla, both the tooth-borne and bone-anchored groups (SNB −2.2° and −1.3°, respectively) enhanced Wits evaluation and ANB changes. The miniscrews, on the other hand, help to limit the downward movement of the maxilla and, as a result, the clock-wise rotation of the mandible in the bone-anchored group. Furthermore, the bone-anchored group had more downward movement of the maxillary incisors than the tooth-borne group, contributing to the bone-anchored group’s preservation of the OVB.
In Willman et al. [53] study positive skeletal modifications rather than dentoalveolar compensation were shown to be responsible for the majority of the overjet correction (FM group 3.51 mm, ME group 3.06 mm). While the skeletal effects on the maxilla were similar in both groups, the SNB angle was significantly reduced. In the FM group, the interbase angle was observed to be more opened, caused by a posterior rotation of the mandible, which might be due to the chincup of the FM. The B point remains constant in the MP-group, while the gonial angle declines considerably, owing to condylar development redirection [53].

5. Conclusions

According to the nine studies included in this systematic review, combining tooth-borne and bone-borne appliances for rapid maxillary extension may be advised in treatment protocols for skeletal Class III patients to obtain more skeletal results while lowering maxillary dentition side effects.

Author Contributions

Conceptualization, V.G. and L.P.; methodology, F.d. and L.N.; data curation, B.M.S. and G.M.; writing—original draft preparation, V.G. and L.N.; writing—review and editing, F.d. and A.D.I.; visualization, G.D.; supervision, L.P. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Please contact the corresponding author.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Ngan, P.; Moon, W. Evolution of Class III Treatment in Orthodontics. Am. J. Orthod. Dentofacial Orthop. 2015, 148, 22–36. [Google Scholar] [CrossRef] [Green Version]
  2. Duggal, R.; Mathur, V.; Parkash, H.; Jena, A. Class—III Malocclusion: Genetics or Environment? A Twins Study. J. Indian Soc. Pedod. Prev. Dent. 2005, 23, 27. [Google Scholar] [CrossRef] [PubMed]
  3. Adina, S.; Dipalma, G.; Bordea, I.R.; Lucaciu, O.; Feurdean, C.; Inchingolo, A.D.; Septimiu, R.; Malcangi, G.; Cantore, S.; Martin, D.; et al. Orthopedic Joint Stability Influences Growth and Maxillary Development: Clinical Aspects. J. Biol. Regul. Homeost. Agents 2020, 34, 747–756. [Google Scholar] [CrossRef]
  4. Al-Mozany, S.A.; Dalci, O.; Almuzian, M.; Gonzalez, C.; Tarraf, N.E.; Ali Darendeliler, M. A Novel Method for Treatment of Class III Malocclusion in Growing Patients. Prog. Orthod. 2017, 18, 40. [Google Scholar] [CrossRef] [Green Version]
  5. Di Venere, D.; Nardi, G.M.; Lacarbonara, V.; Laforgia, A.; Stefanachi, G.; Corsalini, M.; Grassi, F.R.; Rapone, B.; Pettini, F. Early mandibular canine-lateral incisor transposition: Case Report. Oral Implantol. 2017, 10, 181–189. [Google Scholar] [CrossRef] [PubMed]
  6. Di Venere, D.; Corsalini, M.; Nardi, G.M.; Laforgia, A.; Grassi, F.R.; Rapone, B.; Pettini, F. Obstructive site localization in patients with Obstructive Sleep Apnea Syndrome: A comparison between otolaryngologic data and cephalometric values. Oral Implantol. 2017, 10, 295–310. [Google Scholar] [CrossRef] [PubMed]
  7. Di Venere, D.; Pettini, F.; Nardi, G.M.; Laforgia, A.; Stefanachi, G.; Notaro, V.; Rapone, B.; Grassi, F.R.; Corsalini, M. Correlation between parodontal indexes and orthodontic retainers: Prospective study in a group of 16 patients. Oral Implantol. 2017, 10, 78–86. [Google Scholar] [CrossRef]
  8. Cantore, S.; Ballini, A.; Farronato, D.; Malcangi, G.; Dipalma, G.; Assandri, F.; Garagiola, U.; Inchingolo, F.; De Vito, D.; Cirulli, N. Evaluation of an oral appliance in patients with mild to moderate obstructive sleep apnea syndrome intolerant to continuous positive airway pressure use: Preliminary results. Int. J. Immunopathol. Pharmacol. 2016, 29, 267–273. [Google Scholar] [CrossRef] [Green Version]
  9. Dimonte, M.; Inchingolo, F.; Minonne, A.; Arditi, G.; Dipalma, G. Bone SPECT in management of mandibular condyle hyperplasia. Report of a case and review of literature. Minerva Stomatol. 2004, 53, 281–285. [Google Scholar]
  10. Sirbu, A.A.; Bordea, R.; Lucaciu, O.; Braitoru, C.; Szuhanek, C.; Campian, R. 3D Printed Splints an Innovative Method to Treat Temporomandibular Joint Pathology. Rev. Chim. 2018, 69, 3087–3089. [Google Scholar] [CrossRef]
  11. Laudadio, C.; Inchingolo, A.D.; Malcangi, G.; Limongelli, L.; Marinelli, G.; Coloccia, G.; Montenegro, V.; Patano, A.; Inchingolo, F.; Bordea, I.R.; et al. Management of Anterior Open-Bite in the Deciduous, Mixed and Permanent Dentition Stage: A Descriptive Review. J. Biol. Regul. Homeost. Agents 2021, 35, 271–281. [Google Scholar] [CrossRef] [PubMed]
  12. Coloccia, G.; Inchingolo, A.D.; Inchingolo, A.M.; Malcangi, G.; Montenegro, V.; Patano, A.; Marinelli, G.; Laudadio, C.; Limongelli, L.; Di Venere, D.; et al. Effectiveness of Dental and Maxillary Transverse Changes in Tooth-Borne, Bone-Borne, and Hybrid Palatal Expansion through Cone-Beam Tomography: A Systematic Review of the Literature. Medicina 2021, 57, 288. [Google Scholar] [CrossRef] [PubMed]
  13. Patano, A.; Cirulli, N.; Beretta, M.; Plantamura, P.; Inchingolo, A.D.; Inchingolo, A.M.; Bordea, I.R.; Malcangi, G.; Marinelli, G.; Scarano, A.; et al. Education Technology in Orthodontics and Paediatric Dentistry during the COVID-19 Pandemic: A Systematic Review. Int. J. Environ. Res. Public. Health 2021, 18, 6056. [Google Scholar] [CrossRef] [PubMed]
  14. Montenegro, V.; Inchingolo, A.D.; Malcangi, G.; Limongelli, L.; Marinelli, G.; Coloccia, G.; Laudadio, C.; Patano, A.; Inchingolo, F.; Bordea, I.R.; et al. Compliance of Children with Removable Functional Appliance with Microchip Integrated during Covid-19 Pandemic: A Systematic Review. J. Biol. Regul. Homeost. Agents 2021, 35, 365–377. [Google Scholar] [CrossRef]
  15. Almuzian, M.; Almukhtar, A.; Ulhaq, A.; Alharbi, F.; Darendeliler, M.A. 3D Effects of a Bone-Anchored Intra-Oral Protraction in Treating Class III Growing Patient: A Pilot Study. Prog. Orthod. 2019, 20, 37. [Google Scholar] [CrossRef]
  16. Grassia, V.; d’Apuzzo, F.; DiStasio, D.; Jamilian, A.; Lucchese, A.; Perillo, L. Upper and Lower Arch Changes after Mixed Palatal Expansion Protocol. Eur. J. Paediatr. Dent. 2014, 15, 375–380. [Google Scholar]
  17. Page, M.J.; McKenzie, J.E.; Bossuyt, P.M.; Boutron, I.; Hoffmann, T.C.; Mulrow, C.D.; Shamseer, L.; Tetzlaff, J.M.; Akl, E.A.; Brennan, S.E.; et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ 2021, 71, 372. [Google Scholar] [CrossRef]
  18. Alzabibi, B.A.; Burhan, A.S.; Hajeer, M.Y.; Nawaya, F.R. Short-term effects of the orthodontic removable traction appliance in the treatment of skeletal Class III malocclusion: A randomized controlled trial. Dent. Med. Probl. 2021, 58, 163–172. [Google Scholar] [CrossRef]
  19. Fatima, F.; Jeelani, W.; Ahmed, M. Current trends in craniofacial distraction: A literature review. Dent. Med. Probl. 2020, 57, 441–448. [Google Scholar] [CrossRef]
  20. Fabozzi, F.F.; Nucci, L.; Correra, A.; d’Apuzzo, F.; Franchi, L.; Perillo, L. Comparison of Two Protocols for Early Treatment of Dentoskeletal Class III Malocclusion: Modified SEC III versus RME/FM. Orthod. Craniofac. Res. 2021, 24, 344–350. [Google Scholar] [CrossRef]
  21. Inchingolo, A.D.; Patano, A.; Coloccia, G.; Ceci, S.; Inchingolo, A.M.; Marinelli, G.; Malcangi, G.; Montenegro, V.; Laudadio, C.; Palmieri, G.; et al. Genetic Pattern, Orthodontic and Surgical Management of Multiple Supplementary Impacted Teeth in a Rare, Cleidocranial Dysplasia Patient: A Case Report. Medicina 2021, 57, 1350. [Google Scholar] [CrossRef] [PubMed]
  22. Giugliano, D.; d’Apuzzo, F.; Majorana, A.; Campus, G.; Nucci, F.; Flores-Mir, C.; Perillo, L. Influence of occlusal characteristics, food intake and oral hygiene habits on dental caries in adolescents: A cross-sectional study. Eur. J. Paediatr. Dent. 2018, 19, 95–100. [Google Scholar] [CrossRef] [PubMed]
  23. Franchi, L.; Baccetti, T.; McNamara, J.A. Postpubertal Assessment of Treatment Timing for Maxillary Expansion and Protraction Therapy Followed by Fixed Appliances. Am. J. Orthod. Dentofacial Orthop. 2004, 126, 555–568. [Google Scholar] [CrossRef] [PubMed]
  24. Patianna, A.G.; Ballini, A.; Meneghello, M.; Cantore, S.; Inchingolo, A.M.; Dipalma, G.; Inchingolo, A.D.; Inchingolo, F.; Malcangi, G.; Lucchese, A.; et al. Comparison of conventional orthognathic surgery and “surgery-first” protocol: A new weapon against time. J. Biol. Regul. Homeost. Agents 2019, 33, 59–67. [Google Scholar] [PubMed]
  25. Quaglia, E.; Moscufo, L.; Corsalini, M.; Coscia, D.; Sportelli, P.; Cantatore, F.; De Rinaldis, C.; Rapone, B.; Carossa, M.; Carossa, S. Polyamide vs silk sutures in the healing of postextraction sockets: A split mouth study. Oral Implantol. 2018, 11, 115–120. [Google Scholar]
  26. Rapone, B.; Ferrara, E.; Corsalini, M.; Converti, I.; Grassi, F.R.; Santacroce, L.; Topi, S.; Gnoni, A.; Scacco, S.; Scarano, A.; et al. The Effect of Gaseous Ozone Therapy in Conjunction with Periodontal Treatment on Glycated Hemoglobin Level in Subjects with Type 2 Diabetes Mellitus: An Unmasked Randomized Controlled Trial. Int. J. Environ. Res. Public Health 2020, 17, 54675. [Google Scholar] [CrossRef]
  27. Farronato, M.; Farronato, D.; Inchingolo, F.; Grassi, L.; Lanteri, V.; Maspero, C. Evaluation of Dental Surface after De-Bonding Orthodontic Bracket Bonded with a Novel Fluorescent Composite: In Vitro Comparative Study. Appl. Sci. 2021, 11, 6354. [Google Scholar] [CrossRef]
  28. Inchingolo, F.; Tatullo, M.; Marrelli, M.; Inchingolo, A.M.; Tarullo, A.; Inchingolo, A.D.; Dipalma, G.; Brunetti, S.P.; Tarullo, A.; Cagiano, R. Combined Occlusal and Pharmacological Therapy in the Treatment of Temporo-Mandibular Disorders. Eur. Rev. Med. Pharmacol. Sci. 2011, 15, 1296–1300. [Google Scholar]
  29. Wilmes, B.; Nienkemper, M.; Drescher, D. Application and Effectiveness of a Mini-Implant- and Tooth-Borne Rapid Palatal Expansion Device: The Hybrid Hyrax. World J. Orthod. 2010, 11, 323–330. [Google Scholar]
  30. Grassi, F.R.; Grassi, R.; Rapone, B.; Gianfranco, A.; Balena, A.; Kalemaj, Z. Dimensional changes of buccal bone plate in immediate implants inserted through open flap, open flap and bone grafting, and flapless technique. A CBCT randomized controlled clinical trial. Clin Oral Implants Res. 2019, 30, 1155–1164. [Google Scholar] [CrossRef]
  31. Rapone, B.; Corsalini, M.; Converti, I.; Loverro, M.T.; Gnoni, A.; Trerotoli, P.; Ferrara, E. Does Periodontal Inflammation Affect Type 1 Diabetes in Childhood and Adolescence? A Meta-Analysis. Front. Endocrinol. 2020, 11, 278. [Google Scholar] [CrossRef] [PubMed]
  32. Ballini, A.; Cantore, S.; Scacco, S.; Perillo, L.; Scarano, A.; Aityan, S.K.; Contaldo, M.; Nguyen, K.C.; Santacroce, L.; Syed, J.; et al. A comparative study on different stemness gene expression between dental pulp stem cells vs. dental bud stem cells. Eur. Rev. Med. Pharmacol. Sci. 2019, 23, 1626–1633. [Google Scholar] [PubMed]
  33. Grassi, F.R.; Rapone, B.; Scarano Catanzaro, F.; Corsalini, M.; Kalemaj, Z. Effectiveness of computer-assisted anesthetic delivery system (STA™) in dental implant surgery: A prospective study. Oral Implantol. 2017, 10, 381–389. [Google Scholar] [CrossRef] [PubMed]
  34. Corsalini, M.; Di Venere, D.; Carossa, M.; Ripa, M.; Sportelli, P.; Cantatore, F.; De Rinaldis, C.; Di Santantonio, G.; Lenoci, G.; Barile, G.; et al. Comparative clinical study between zirconium-ceramic and metal-ceramic fixed rehabilitations. Oral Implantol. 2018, 11, 150–160. [Google Scholar]
  35. Ballini, A.; Cantore, S.; Fotopoulou, E.A.; Georgakopoulos, I.P.; Athanasiou, E.; Bellos, D.; Paduanelli, G.; Saini, R.; Dipalma, G.; Inchingolo, F. Combined sea salt-based oral rinse with xylitol in orthodontic patients: Clinical and microbiological study. J. Biol. Regul. Homeost. Agents 2019, 33, 263–268. [Google Scholar]
  36. Rapone, B.; Ferrara, E.; Santacroce, L.; Cesarano, F.; Arazzi, M.; Di Liberato, L.; Scacco, S.; Grassi, R.; Grassi, F.R.; Gnoni, A.; et al. Periodontal Microbiological Status Influences the Occurrence of Cyclosporine-A and Tacrolimus- Induced Gingival Overgrowth. Antibiotics 2019, 8, 124. [Google Scholar] [CrossRef] [Green Version]
  37. Corsalini, M.; Di Venere, D.; Sportelli, P.; Magazzino, D.; Ripa, M.; Cantatore, F.; Cagnetta, C.; De Rinaldis, C.; Montemurro, N.; De Giacomo, A.; et al. Evaluation of prosthetic quality and masticatory efficiency in patients with total removable prosthesis: Study of 12 cases. Oral Implantol. 2018, 11, 230–240. [Google Scholar]
  38. Cantore, S.; Ballini, A.; De Vito, D.; Martelli, F.S.; Georgakopoulos, I.; Almasri, M.; Dibello, V.; Altini, V.; Farronato, G.; Dipalma, G.; et al. Characterization of human apical papilla-derived stem cells. J. Biol. Regul. Homeost. Agents 2017, 31, 901–910. [Google Scholar]
  39. Marinelli, G.; Inchingolo, A.D.; Inchingolo, A.M.; Malcangi, G.; Limongelli, L.; Montenegro, V.; Coloccia, G.; Laudadio, C.; Patano, A.; Inchingolo, F.; et al. White spot lesions in orthodontics: Prevention and treatment. A descriptive review. J. Biol. Regul. Homeost. Agents 2021, 35, 227–240. [Google Scholar]
  40. Cirulli, N.; Ballini, A.; Cantore, S.; Farronato, D.; Inchingolo, F.; Dipalma, G.; Gatto, M.R.; Alessandri Bonetti, G. Mixed dentition space analysis of a southern italian population: New regression equations for unerupted teeth. J. Biol. Regul. Homeost. Agents 2015, 29, 515–520. [Google Scholar]
  41. Cevidanes, L.; Baccetti, T.; Franchi, L.; McNamara, J.A.; De Clerck, H. Comparison of Two Protocols for Maxillary Protraction: Bone Anchors versus Face Mask with Rapid Maxillary Expansion. Angle Orthod. 2010, 80, 799–806. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  42. Heymann, G.C.; Cevidanes, L.; Cornelis, M.; De Clerck, H.J.; Tulloch, J.F.C. Three-Dimensional Analysis of Maxillary Protraction with Intermaxillary Elastics to Miniplates. Am. J. Orthod. Dentofacial Orthop. 2010, 137, 274–284. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  43. Scarano, A.; Inchingolo, F.; Rapone, B.; Festa, F.; Tari, S.R.; Lorusso, F. Protective Face Masks: Effect on the Oxygenation and Heart Rate Status of Oral Surgeons during Surgery. Int. J. Environ. Res. Public Health 2021, 18, 2363. [Google Scholar] [CrossRef] [PubMed]
  44. Lorusso, F.; Noumbissi, S.; Inchingolo, F.; Rapone, B.; Khater, A.G.A.; Scarano, A. Scientific Trends in Clinical Research on Zirconia Dental Implants: A Bibliometric Review. Materials 2020, 13, 5534. [Google Scholar] [CrossRef] [PubMed]
  45. Bordea, I.; Sîrbu, A.; Lucaciu, O.; Ilea, A.; Câmpian, R.; Todea, D.; Alexescu, T.; Aluaș, M.; Budin, C.; Pop, A. Microleakage—The Main Culprit in Bracket Bond Failure? J. Mind Med. Sci. 2019, 6, 86–94. [Google Scholar] [CrossRef] [Green Version]
  46. Moon, W.; Wu, K.W.; MacGinnis, M.; Sung, J.; Chu, H.; Youssef, G.; Machado, A. The Efficacy of Maxillary Protraction Protocols with the Micro-Implant-Assisted Rapid Palatal Expander (MARPE) and the Novel N2 Mini-Implant—a Finite Element Study. Prog. Orthod. 2015, 16, 16. [Google Scholar] [CrossRef] [Green Version]
  47. Ottawa Hospital Research Institute. Available online: http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp (accessed on 26 January 2022).
  48. Montinaro, F.; Nucci, L.; Carfora, M.; d’Apuzzo, F.; Franchi, L.; Perillo, L. Modified SEC III Protocol: Vertical Control Related to Patients’ Compliance with the Chincup. Eur. J. Orthod. 2021, 43, 80–85. [Google Scholar] [CrossRef]
  49. Maino, G.; Turci, Y.; Arreghini, A.; Paoletto, E.; Siciliani, G.; Lombardo, L. Skeletal and Dentoalveolar Effects of Hybrid Rapid Palatal Expansion and Facemask Treatment in Growing Skeletal Class III Patients. Am. J. Orthod. Dentofacial Orthop. 2018, 153, 262–268. [Google Scholar] [CrossRef]
  50. Hourfar, J.; Kinzinger, G.S.M.; Ludwig, B.; Spindler, J.; Lisson, J.A. Differential Treatment Effects of Two Anchorage Systems for Rapid Maxillary Expansion: A Retrospective Cephalometric Study. J. Orofac. Orthop. Fortschr. Kieferorthopädie 2016, 77, 314–324. [Google Scholar] [CrossRef]
  51. Nienkemper, M.; Wilmes, B.; Franchi, L.; Drescher, D. Effectiveness of Maxillary Protraction Using a Hybrid Hyrax-Facemask Combination: A Controlled Clinical Study. Angle Orthod. 2015, 85, 764–770. [Google Scholar] [CrossRef] [Green Version]
  52. Marra, P.M.; Nucci, L.; Abdolreza, J.; Perillo, L.; Itro, A.; Grassia, V. Odontoma in a young and anxious patient associated with unerupted permanent mandibular cuspid: A case report. J. Int. Oral Health 2020, 12, 182–186. [Google Scholar] [CrossRef]
  53. Willmann, J.H.; Nienkemper, M.; Tarraf, N.E.; Wilmes, B.; Drescher, D. Early Class III Treatment with Hybrid-Hyrax -Facemask in Comparison to Hybrid-Hyrax-Mentoplate—Skeletal and Dental Outcomes. Prog. Orthod. 2018, 19, 42. [Google Scholar] [CrossRef] [PubMed]
  54. Ngan, P.; Wilmes, B.; Drescher, D.; Martin, C.; Weaver, B.; Gunel, E. Comparison of Two Maxillary Protraction Protocols: Tooth-Borne versus Bone-Anchored Protraction Facemask Treatment. Prog. Orthod. 2015, 16, 26. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  55. Seiryu, M.; Ida, H.; Mayama, A.; Sasaki, S.; Sasaki, S.; Deguchi, T.; Takano-Yamamoto, T. A Comparative Assessment of Orthodontic Treatment Outcomes of Mild Skeletal Class III Malocclusion Between Facemask and Facemask in Combination with a Miniscrew For Anchorage in Growing Patients: A Single-Center, Prospective Randomized Controlled Trial. Angle Orthod. 2020, 90, 3–12. [Google Scholar] [CrossRef] [Green Version]
  56. Dekel, E.; Nucci, L.; Weill, T.; Flores-Mir, C.; Becker, A.; Perillo, L.; Chaushu, S. Impaction of Maxillary Canines and Its Effect on the Position of Adjacent Teeth and Canine Development: A Cone-Beam Computed Tomography Study. Am. J. Orthod. Dentofacial Orthop. 2021, 159, e135–e147. [Google Scholar] [CrossRef]
  57. Eslami, S.; Faber, J.; Fateh, A.; Sheikholaemmeh, F.; Grassia, V.; Jamilian, A. Treatment Decision in Adult Patients with Class III Malocclusion: Surgery versus Orthodontics. Prog. Orthod. 2018, 19, 28. [Google Scholar] [CrossRef] [Green Version]
  58. Clemente, R.; Contardo, L.; Greco, C.; Di Lenarda, R.; Perinetti, G. Class III Treatment with Skeletal and Dental Anchorage: A Review of Comparative Effects. BioMed Res. Int. 2018, 2018. [Google Scholar] [CrossRef]
  59. Luchian, I.; Moscalu, M.; Goriuc, A.; Nucci, L.; Tatarciuc, M.; Martu, I.; Covasa, M. Using Salivary MMP-9 to Successfully Quantify Periodontal Inflammation during Orthodontic Treatment. J. Clin. Med. 2021, 10, 379. [Google Scholar] [CrossRef]
  60. De Felice, M.E.; Nucci, L.; Fiori, A.; Flores-Mir, C.; Perillo, L.; Grassia, V. Accuracy of interproximal enamel reduction during clear aligner treatment. Prog. Orthod. 2020, 21, 28. [Google Scholar] [CrossRef]
  61. Inchingolo, A.D.; Patano, A.; Coloccia, G.; Ceci, S.; Inchingolo, A.M.; Marinelli, G.; Malcangi, G.; Di Pede, C.; Garibaldi, M.; Ciocia, A.M.; et al. Treatment of Class III Malocclusion and Anterior Crossbite with Aligners: A Case Report. Medicina 2022, 58, 603. [Google Scholar] [CrossRef]
Applsci 12 04646 g001 550
Figure 1. Risk of bias assessed according to Cochrane.
Figure 1. Risk of bias assessed according to Cochrane.
Applsci 12 04646 g001
Applsci 12 04646 g002 550
Figure 2. Flow diagram of the selection of the studies (according to PRISMA 2020 flow diagram) [17].
Figure 2. Flow diagram of the selection of the studies (according to PRISMA 2020 flow diagram) [17].
Applsci 12 04646 g002
Table
Table 1. Studies’ quality according to the Newcastle–Ottawa Scale.
Table 1. Studies’ quality according to the Newcastle–Ottawa Scale.
Author/Year [Reference]SelectionComparabilityExposure
Case Definition AdequateRepresentativeness of CasesSelection of ControlsDefinition of ControlsComparability of Cases & ControlsAscertainment of ExposureSame Method of AscertainmentNon-Response Rate
Al-Mozany et al.
Almuzian et al.		 * **
Wilmes et al. **
Maino et al. * **
Nienkemper et al.*******
Hourfar et al. *****
Seiryu et al.* *****
Ngan et al. ******
Willmann et al. ******
Table
Table 2. Studies involved in the qualitative analysis.
Table 2. Studies involved in the qualitative analysis.
AuthorsAimC (Cases) Co (Controls)Mean Age (MA)RandomizedControl
(Yes or Not)		Number and Position of Screw (Appliance Design)Inclusion CriteriaMethodsResults
Al-Mozany et al./
Almuzian et al.		To evaluate the skeletal and soft tissue effects of the alternate RME and constriction (Alt-Ramec) protocol in conjunction with a miniscrew forgrowing participants with retrognathic maxilla, evaluated by cephalometric analysis (2017) and 3D cone-beam (2019).Ca (14)
MA (12.5 ± 1.9 years)		NoNoMARME with two paramedial palatal TADs and two mandibular TADs, inserted between the canine and the lateral incisor-Patients at CVM Stage 2 or 3
-Patients with clinically diagnosed retrognathic or hypoplastic maxilla
-Anterior crossbite and dental Class III molars and canines		All participants had a MARME appliance that was activated by the Alt-Ramec protocol. Full time Class III elastics, delivering 400 g/side, were then used for maxillary protraction.-Maxilla protraction (SNA 1.87 ± 1.06°; Vert.T-A 3.29 ± 1.54 mm p < 0.001)
-Mandibular retropositioning (SNB-2.03 ± 0.85°; Vert.T-B-3.43 ± 4.47 mm, p < 0.001 and p < 0.05 respectively)
-A significant improvement in the skeletal relationship (ANB 3.95 ± 0.57°, p < 0.001; Wits 5.15 ±1.51 mm, p < 0.001)
-Increase of Y-axis angle (1.95 ± 1.11°, p < 0.001)
-The upper incisors proclination (+ 2.98 ± 2.71°, p < 0.01), coupled with a significant retroclination of the lower incisors (−3.2 ± 3.4°, p < 0.05)
-A significant improvement in the OVJ (5.62 ± 1.36 mm, p < 0.001) and in the Harmony angle (2.76 ± 1.8°, p < 0.001)
Wilmes
et al.		To assess the clinical applicability and 3D effects of RPE using the hybrid hyrax.Ca (13)
MA (11.2 years)		NoNoTwo miniscrews were inserted in the anterior palate next to the midpalatal suture and near the second and third palatal rugae. The miniscrews and two molar bands were used to connect the Hybrid Hyrax device.-Patients with Class III malocclusionRPE was performed in 13 patients. In 10 patients with a skeletal Class III occlusion, a facemask was used for maxillary protraction.-The mean expansion in the first premolar/first primary molar region was 6.3 ± 2.9 mm and 5.0 ± 1.5 mm in the first molar region.
-The Wits appraisal changed from −5.2 ± 1.3 mm to −2.5 ± 1.5 mm (mean improvement 2.7 ± 1.3 mm).
-The right first molar migrated 0.4 ± 0.6 mm mesially and the left one 0.3 ± 0.2 mm.
Maino et al.To describe the skeletal and dentoalveolar changes in a group of growing skeletal Class III patients treated with hybrid palatal expander and facemask.Ca (28)
MA (11.4 ± 2.5 years)		NoNoHybrid palatal expander was connected to two paramedial palatal miniscrews-Growing patient with Class III malocclusion.28 patients were treated using a rapid maxillary expander with hybrid anchorage according to the ALT-Ramec protocol, followed by 4 months of facemask therapy. Palatal miniscrew placement was accomplished via digital planning and the construction of a high-precision, individualized surgical guide.-Point A advanced by a mean of 3.4 mm with respect to the reference plane Vert-T.
-The mandibular plane rotated clockwise, improving the ANB (+3.41°) and the Wits appraisal (+4.92 mm).
-The maxillary first molar had slight extrusion (0.42 mm) and mesialization (0.87 mm).
Nienkemper et al.To value the efficacy of hybrid hyrax and facemask (FM) combination in growing Class III patients.Ca (16)
MA (9.5 ± 1.6 years)
Co (16)
MA (9.4 ± 1.1 years)		NoYesOn both sides of the midpalatal suture, two micro implants were placed in the anterior palate. The miniscrews and two molar bands were used to connect the Hybrid hyrax device. To apply the protraction, two rigid sectional wires were welded to the buccal side of the molar bands.-Class III malocclusion in the mixed dentition characterized by a Wits index of −2 mm or less (mean, −5.6 ± 2.2 mm)
-Anterior crossbite or incisor edge-to-edge relationship
-Class III molar relationship
-CVM (CS1-3)		16 growing Class III patients were treated with a RPE with a hybrid hyrax. A facemask was used to perform a maxillary protraction. A control group of 16 untreated Class III participants was compared to the treatment group.-SNA and Point A to nasion perpendicular showed significant increments of 2.4° and 2.4 mm, respectively.
-A significant reduction in the length and sagittal position of the mandible (CoGn −2.3 mm and SNB −1.7°)
-The Wits index augmented by 4.5 mm and the ANB improved by 4.1°
-Co-Go-Me angle decrease significantly (2.0°). No significant increase of FMA (0.5°)
-OVJ and molar relationship improved significantly (3.2 mm and −3.1 mm, respectively)
-No significant differences could be found in OVB or inclination of the maxillary and mandibular incisors.
Hourfar et al.To compare cephalometric changes after treatment with two types of fast maxillary expansion appliances: a tooth-borne appliance and a tooth-and-bone-borne appliance.Ca (50);
MA (13.04 ± 4.82 years)
Co (50);
MA (13.04 ± 4.82 years)		NoYesTwo miniscrews were placed in the anterior palate at paramedian locations. These miniscrews held the anterior side of a hybrid hyrax, while the posterior side was attached to the orthodontic bands on the front molars.-Patients treated with strictly tooth-borne or patients treated with combined tooth- and bone-borne appliances
-Skeletal Class I (0° < ANB ≤ 4°) or Class III (ANB ≤ 0°)
-Caucasian descendent
-Bilateral posterior crossbite		Cephalometry was used to examine the pre- and post-treatment lateral cephalograms of 100 patients. 50 of the patients were treated with exclusively tooth-borne appliances, whereas the other 50 were treated with a combination of tooth-and-bone-borne appliances. Additional sub-groups were constructed based on pre-treatment cephalometric data for skeletal Class I or Class III to detect any implications for clinical therapy.-Pronounced anterior shift of the maxilla (SNA increase by 2°)
-Caudal shift of the maxilla
-Upper jaw inclination remained almost the same.
-Increases in the vertical parameters ML/NSL (1.46°) and Björk sum (1.46).
Seiryu et al.To investigate difference in treatment outcomes of milder skeletal class III malocclusion between FM and FM + MS in growing patientsCa (20)
MA (10.05 ± 1.8 years)
Co (19)
MA (11 ± 1.3 years)		YesYesIn the FM + MS group, one miniscrew was inserted in the anterior region of the palate. A lingual arch with soldered hooks was attached to the miniscrew.-Skeletal class III (ANB ≤ 2.5°)
-OVJ ≥ 0
-CVMS II–IV
-No congenital or systemic disease
-No skeletal asymmetry
-No missing teeth
-No temporomandibular joint disorder		A lingual arch with hooks was fixed to the maxillary arch in both groups and a protractive force of 500 g was applied form the FM to the hooks, 12 hours per day.-No MS mobility or loss during treatment
-Cephalometric analysis showed a significant increase In SNA (1.1°), SN-ANS (1.3°), and ANB (0.8°) in the FM + MS group
-Proclination of maxillary incisors significantly greater in the FM group (4.6°)
Ngan et al.The goal of this study was to examine the skeletal and dentoalveolar alterations in patients treated with tooth-borne Hyrax + FM versus hybrid Hyrax + FM.Ca (20)
MA (9.8 ± 1.6 years)
Co (20)
MA (9.6 ± 1.2 years)		NoYesIn the hybrid Hyrax group two Benefit micro-implants were placed in the third palatal rugae.-Class III malocclusion:
-Anterior crossbite or edge-to-edge incisal relationship
-Wits ≤ −3/ANB ≤ −2°		A total of 20 patients were treated with tooth-borne Hyrax + FM, while 20 patients were treated with bone-borne Hyrax +FM in a row. The screw was activated twice daily by the patients in both groups for one week, and two weeks when a constricted maxilla was evident. Maxillary protraction was performed with 380 g per side elastics for 12–14 hours per day.-Sagittal relationship improved in both groups: tooth-borne (Wits 2.19, ANB 2.58°) and bone-borne (Wits 2.31 mm, ANB 2.17°).
-Greater downward movement of the maxilla in tooth-borne (OLparallel–A pt. 1.2 mm) compared to bone-borne (−0.4 mm, p < 0.005)
-Forward and backward movement of the mandible in both groups (OLp-A pt., 0.7 mm/2.2 mm)
-Mandibular plane angle was found to open more in tooth-borne (SNL-ML 2.76°) compared to bone-borne (−0.25°, p < 0.05)
-Greater change in OVJ in the tooth-borne group (5.5 mm) compared to the bone-anchored group (3.4 mm, p < 0.001)
-Greater forward movement of maxillary incisors in the tooth-borne group (OLp-Is 2.12 vs. 0.87, p < 0.05) and greater differential maxillary/mandibular molars movement
Willmann et al.To compare skeletal and dental effects of hybrid Hyrax + FM and hybrid Hyrax + Mentoplate (MP)Ca (17)
MA (8.74 ± 1.20 years)
Co (17)
MA (9.43 ± 0.95 years)		NoYesHybrid Hyrax devices were fitted on two paramedian mini-implants in the anterior palate; Mentoplate was inserted subapical to the lower incisors.-Wits ≤ −2.0 mm
-From 7 to 12 years
-Anterior crossbite or incisor edge-to-edge relationship
-Molar class III relationship		34 patients were treated with Hybrid Hyrax. In 17, maxillary protraction was performed with FM, while the other 17 were treated in combination with MP. The expander screw was activated 4 times/day; the FM group wore 400 g elastics on each side 14–16 h per day while the MP full-time.-Significant forward movement of A-point (FM GROUP: SNA + 2.23° ± 1.30°— p 0.000 *; ME: 2.23° ± 1.43°— p 0.000 *);
-B-Point showed a larger sagittal change in the FM Group (SNB 1.51 ± 1.1°— p 0.000 *) compared to the ME group (SNB: − 0.30 ± 0.9°— p 0.070);
-FM group showed a significant increase in the ML-NL + 1.86 ± 1.65° (p 0.000 *) and NSL-ML + 1.17 ± 1.48 (p 0.006 *);
-Upper incisor inclination as well as the distance of the first upper Molar in relation to A-point did not change significantly in both groups.
Table
Table 3. Dentoskeletal cephalometric results.
Table 3. Dentoskeletal cephalometric results.
AuthorsSNASNBANBOVBOVJWITSSN^GoMe or EquivalentsUpper Incisor PositionLower Incisor Position
Al-Mozany et al./Almuzian et al.1.87 ± 1.06°2.02 ± 0.85°3.95 ± 0.57°1.21 ± 1.89 mm5.63 ± 1.36 mm5.16 ± 1.5 mmY-axis:1.95° ± 1.11°UI-PP:2.98 ± 2.71°LI-MP:3.2 ± 3.4°
Wilmes et al.nanananana2.7 ± 1.3 mmnanana
Maino et al.+2.50°−0.92°+3.41°nana+4.92 mmSN-GoGn: +1.64°U1-PP: −2.26°
Pr-VertT:
+3.62 mm		na
Nienkemper et al.2.4°−1.7°4.1°−0.2 mm3.2 mm4.5 mmCo-Go-Me:
−2.0°		U1-PP:
−0.5°		L1-MP:
−1.7°
Hourfar et al.2.17°−0.97°+2.77°nananaML/NSL: 1.46°nana
Seiryu et al.+2.2 ± 1.3°+0.1 ± 1.3+2.0 ± 1.3nananaMP-SN: −0.1 ± 1.3U1-SN: −0.4 ± 4.2na
Ngan et al.1.59°−0.8°+2.4°−0.14 mm+3.46 mm+2.58 mmSNL-ML: +0.24°Is-SNL: −2.03Ii-ML: −1.67
Willmann et al.FM: 2.23 ± 1.30
MP: 2.23 ± 1.43		FM:−1.51 ± 1.13
MP: −0.30 ± 0.98		FM: 3.75 ± 1.45
MP: 2.54 ± 0.99		nanaFM: 4.81 ± 1.38
MP: 4.14 ± 1.25		ML-NSL:
FM: 1.17 ± 1.48
MP: − 0.55 ± 1.09		U1-PP°:
FM:−1.15 ± 6.45
MP: 0.57 ± 5.49		L1-ML°:
FM: −3.84 ± 6.13
MP: 0.56 ± 3.83
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

d’Apuzzo, F.; Nucci, L.; Strangio, B.M.; Inchingolo, A.D.; Dipalma, G.; Minervini, G.; Perillo, L.; Grassia, V. Dento-Skeletal Class III Treatment with Mixed Anchored Palatal Expander: A Systematic Review. Appl. Sci. 2022, 12, 4646. https://doi.org/10.3390/app12094646

AMA Style

d’Apuzzo F, Nucci L, Strangio BM, Inchingolo AD, Dipalma G, Minervini G, Perillo L, Grassia V. Dento-Skeletal Class III Treatment with Mixed Anchored Palatal Expander: A Systematic Review. Applied Sciences. 2022; 12(9):4646. https://doi.org/10.3390/app12094646

Chicago/Turabian Style

d’Apuzzo, Fabrizia, Ludovica Nucci, Bruno M. Strangio, Alessio Danilo Inchingolo, Gianna Dipalma, Giuseppe Minervini, Letizia Perillo, and Vincenzo Grassia. 2022. "Dento-Skeletal Class III Treatment with Mixed Anchored Palatal Expander: A Systematic Review" Applied Sciences 12, no. 9: 4646. https://doi.org/10.3390/app12094646

APA Style

d’Apuzzo, F., Nucci, L., Strangio, B. M., Inchingolo, A. D., Dipalma, G., Minervini, G., Perillo, L., & Grassia, V. (2022). Dento-Skeletal Class III Treatment with Mixed Anchored Palatal Expander: A Systematic Review. Applied Sciences, 12(9), 4646. https://doi.org/10.3390/app12094646

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop