Multi-View 3D Transesophageal Echocardiography Registration and Volume Compounding for Mitral Valve Procedure Planning

Abstract

Three-dimensional ultrasound mosaicing can increase image quality and expand the field of view. However, limited work has been done applying these compounded approaches for cardiac procedures focused on the mitral valve. For procedures targeting the mitral valve, transesophageal echocardiography (TEE) is the primary imaging modality used as it provides clear 3D images of the valve and surrounding tissues. However, TEE suffers from image artefacts and signal dropout, particularly for structures lying below the valve, including chordae tendineae, making it necessary to acquire alternative echo views to visualize these structures. Due to the limited field of view obtainable, the entire ventricle cannot be directly visualized in sufficient detail from a single image acquisition in 3D. We propose applying an image compounding technique to TEE volumes acquired from a mid-esophageal position and several transgastric positions in order to reconstruct a high-detail volume of the mitral valve and sub-valvular structures. This compounding technique utilizes both fully and semi-simultaneous group-wise registration to align the multiple 3D volumes, followed by a weighted intensity compounding step based on the monogenic signal. This compounding technique is validated using images acquired from two excised porcine mitral valve units and three patient data sets. We demonstrate that this compounding technique accurately captures the physical structures present, including the mitral valve, chordae tendineae and papillary muscles. The chordae length measurement error between the compounded ultrasound and ground-truth CT for two porcine valves is reported as 0.7 ± 0.6 mm and 0.6 ± 0.6 mm.

1. Introduction

Three-dimensional (3D) ultrasound imaging is used extensively as a diagnostic and guidance tool for cardiac procedures. Three-dimensional echocardiography allows for the acquisition of volumetric data of the heart, which can be analysed in any plane. The current standard of care for mitral valve procedures includes diagnostic imaging with a 3D transesophageal echocardiography (TEE) probe [1,2]. This method of imaging provides a clear view of the mitral valve, and including color Doppler allows the cardiologist/cardiac surgeon to identify the mitral valve pathology. While echocardiography is a powerful imaging technique, it nevertheless has some major limitations. The field of view is limited when using 3D transducers, which can limit the range of anatomy that can be easily viewed, and structures further away from the image probe may suffer from poor spatial resolution, while thin structures parallel to the ultrasound beam suffer from signal dropout artefacts.

Ultrasound compounding, or mosaicing, has been proposed by several groups to address limitations of 3D ultrasound and improve imaging capability. By registering and blending together adjacent acquisitions from different poses, we can expand the field of view and address the issue of signal dropout, producing higher quality images with greater information for the clinician. Several image compounding techniques have been proposed to register a set of ultrasound volumes, all of which demonstrate improved image quality [3] and provide an avenue for combining common cardiac ultrasound views into a single volume with reduced noise, reduced speckle, and fewer signal dropout artefacts. Researchers have demonstrated 3D ultrasound compounding techniques with applications in cardiac, fetal and breast imaging [4,5,6]. Common across all compounding methods are two critical steps: global registration of all volumes and blending the overlapping regions of the registered volumes to generate the resulting image [3]. Evaluation of registration frameworks has identified three main approaches, consisting of sequential alignment, semi-simultaneous and fully simultaneous registration [7]. Using a sequential alignment approach, each acquisition is registered to the next; however, this technique suffers from drift and error accumulation. The semi-simultaneous approach uses each volume as the moving volume in turn and every other volume as fixed for multiple cycles until convergence is met. This approach balances computational complexity, as only the parameters of a single transform need to be considered in optimization; furthermore, it has global alignment because every volume is considered at every step. The final approach, fully simultaneous group-wise registration, optimizes the transformation parameters of all volumes simultaneously, applying a loss function as the sum of pairwise losses. This approach is optimal for registration quality; however, it is limited by computational complexity due to the number of parameters that need to be optimized.

For mitral valve imaging using standard en-face views, the limitations of 3D ultrasound result in the structures beyond the valve, including the chordae tendineae, papillary muscles, and left-ventricular outflow tract being difficult to identify. While imaging from a different position (e.g., transgastric view) can capture these structures, the field of view limits the utility of these images as at these positions the entire mitral valve apparatus cannot be captured in a single volume. Currently, choosing the treatment plan that provides the greatest benefit to the patient is one of the biggest clinical challenges for cardiologists and cardiac surgeons [8]. Determining optimal neochord length is one of the main issues that cardiac surgeons must address in MV repair procedures [9], and it is particularly challenging to define this length due to a general lack of accurate anatomical information from standard diagnostic imaging, which includes only the en-face view of the mitral valve. Chordae tendineae length measurements are required for mitral valve procedures involving the implantation of artificial chordae [10]. It is nevertheless challenging to determine their optimal length because direct observation is limited to physical measurements made inside the flaccid heart during surgery, along with 2D transgastric long-axis images which require that the image plane be aligned with the entire chord to achieve accurate results [11]. While 3D transgastric TEE ultrasound is safe and easily acquired during routine TEE imaging (adding approximately 3–5 min to the procedure), it is rarely employed due to the limited visibility of the leaflets and field-of-view limitations preventing the entire length of the chordae from being captured. Without the ability to see the entire subvalvular apparatus in the same image data as the standard en-face view of the leaflets, transgastric image information is of very limited clinical value.

In our prior work on volume compounding using TEE volumes, we explored a workflow using only the semi-simultaneous registration strategy and a distance-based weighted average blending [12]. Our prior approach was able to successfully produce compounded volumes; however, the registration component of the workflow was not robust, could fail depending on the order in which the volumes were processed, and the blending approach induced imaging artefacts. In this study, we propose an improved method to register and compound transgastric and en-face volumes utilizing a combination of semi and fully simultaneous registration with a novel weighting function for blending overlapping regions to reduce compounding artefacts. This provides an avenue for combining common cardiac ultrasound views into a single volume with reduced noise and fewer dropout artefacts. Many image compounding techniques involve the use of a tracked probe and were targeted at combining multiple transthoracic views [6]. Our method differs from these previous methods as it does not require any external tracking of the ultrasound probe, and it has been tailored for use with TEE probes to combine 3D mid-esophageal and transgastric volumes that can be acquired as part of a standard diagnostic imaging session. In the en-face volumes, the mitral valve is clearly visible, and in the transgastric views, the chordae are very clear, as these views are nearly perpendicular to each other. By combining both the en-face and transgastric views, we can maintain optimal imaging for both structures in a single compounded volume. Integrated leaflet and chordae geometry in a single volume will greatly improve the cardiac surgeons’ ability to accurately measure the length of individual chordae (a crucial factor in neochordae repair techniques [9]) and plan their repair strategy.

2. Materials and Methods

2.1. Image Registration

Following local REB approval, we adapted standard diagnostic TEE acquisition protocols to include multiple transgastric views in addition to the standard mid-esophageal view. Volumes were acquired using ECG gating to match the cardiac phase. Our imaging protocol requires a minimum of one mid-esophageal acquisition and four transgastric acquisitions with approximately 80% spatial overlap or more between adjacent volumes for successful registration of the acquisitions, an example of which can be seen in Figure 1. The transgastric acquisitions should begin at the mitral valve and proceed along the ventricle to the papillary muscles. Compounding is then accomplished by aligning over-sampled data through automated image registration, re-sampling the aligned volumes into a consistent output space, and generating the output image through the voxel-wise blending of the overlapping volumes. The compounding workflow is shown in Figure 2.

Performing image registration of multiple volumes can be achieved using pairwise, fully simultaneous, or semi-simultaneous approaches [13]. We implemented both the semi-simultaneous and fully simultaneous approaches described by Wachinger et al. [13]. This is done as an extension of our initial work on this method, in which only the semi-simultaneous approach was used [12]. We first perform rigid registration at the end-systole between all volumes using fully simultaneous group-wise registration with the sum of pairwise normalized cross-correlations as the loss function. This gives us a rough global alignment of the volumes and is not dependent on input order. Then, two cycles of semi-simultaneous registration are performed, which we found achieved better agreement between volumes than the fully simultaneous registration alone. Finally, we utilize non-rigid registration in the semi-simultaneous framework at each frame in the acquisitions to account for the slight deviations at different points of the cardiac cycle due to imperfect synchronization. For both semi-simultaneous steps, the loss function used was the sum of normalized cross-correlation between the moving volume and each fixed volume. At each step, optimization was performed using adaptive stochastic gradient descent in a multi-resolution registration framework with four resolution levels, each smoothing the image by a factor of 2 over the previous one. We implemented this approach using the Elastix toolkit (http://elastix.isi.uu.nl/, accessed on10 October 2021) on the 3D Slicer platform (https://www.slicer.org/, accessed on 10 October 2021). This open-source implementation of our work is available at https://github.com/pcarnah/CardiacVolumeStitching, accessed on 10 October 2021.

2.2. Image Blending

After the volumes are registered, re-sampling and compounding are performed at each cardiac phase to construct a 3D + time compounded volume with a wide field of view. Before compounding, the volumes are re-sampled to a common grid using cubic b-spline interpolation to ensure that there is complete voxel overlap between volumes so that the blending step (a weighted average of all overlapping volumes at each voxel location) can be performed. The output grid is determined by the extent of all overlapping input images, using isotropic spacing equal to the minimum spacing in any dimension in any input image.

We evaluated multiple weighting strategies including the voxel-wise maximum, average and weighted average using two different weighting schemes. The voxel weighting methods we compared were the scaled distance from the image probe and a combination of distance from the probe and a feature detector based on the monogenic signal [14]. The local phase measure from the monogenic signal was previously demonstrated as part of an application-specific loss function for ultrasound compounding [15]. For our purposes, we used the oriented symmetry measure, which returns values from $-1$ to $+1$, with positive values being associated with features of interest and negative values being associated with background noise, as visualized in Figure 3. We implemented the 3D extension of the monogenic signal in Python. The distance function assigns higher weights to voxels closer to the image probe, with values ranging from 25 near the probe down to 0 with an inverse-square law dropoff, which we used to approximate the reduction in resolution further away from a phased-array ultrasound probe. For a single voxel position p in a source volume i, the expression for the distance from the image probe weighting $d_i$ is

$$d_i\left(p\right)=25\frac{{\left(10-\frac{10\times \Vert p-o_i\Vert }{{\displaystyle \underset{q\in V_i}{max}\left(\Vert q-o_i\Vert \right)}}\right)}^2}{{10}^2},$$

(1)

where $o_i$ is the position of the probe origin in volume i. The oriented symmetry weighting $S_i$ is given as

$$S_i\left(p\right)=\left\{\begin{array}{c}10\times s_i\left(p\right)\mathrm{if}{s}_i\left(p\right)<0\hfill \\ 25\times s_i\left(p\right)\mathrm{if}{s}_i\left(p\right)\ge 0\hfill \end{array}\right.,$$

(2)

where $s_i$ is the value of the oriented symmetry measure from the monogenic signal at position p in volume i. The combined weighting function $W_i$ of distance and oriented symmetry is

$${\displaystyle W_i\left(p\right)=max\left(0.5,S_i\left(p\right)+d_i\left(p\right)\right),}$$

(3)

giving the sum of the two weights with a minimum value of 0.5. Both the distance weight $d_i$ and symmetry weight $S_i$ have a maximum value of 25, contributing equally to the overall voxel weighting. Finally, the expression for the final weighted average output intensity $F\left(p\right)$ is

$$F\left(p\right)=\frac{{\displaystyle \sum _{V_i|V_i\left(p\right)>0}\left(V_i\left(p\right)\times w_i\left(p\right)\right)}}{{\displaystyle \sum _{V_i|V_i\left(p\right)>0}\left(w_i\left(p\right)\right)}},$$

(4)

where $V_i\left(p\right)$ is the intensity value in volume i at position p, and $w_i$ is either the distance weighting alone or the combined distance and oriented symmetry weighting. A visual comparison of the results of applying the different blending approaches can be seen in Figure 4. The voxel-wise maximum approach produces a volume with very sharp features but passes through any imaging artefacts and highlights registration errors. Simple averaging produces a smoother image but lacks the definition of smaller features and boundary edges appear blurred. Distance weighted averaging further improves image quality, taking advantage of the increased line density nearer to the probe and the corresponding increase in spatial resolution, but small features and edge boundaries are still blurred and lack contrast to the background. The incorporation of the monogenic signal-based feature detector into the weighting function helps to reduce this blurring and makes the structures of interest more distinct without amplifying imaging and registration artefacts.

2.3. Data Acquisition

Three patients were imaged using our acquisition protocol under REB approval, using the Philips Epiq TEE system. These image sets were then registered and combined with each of the four blending approaches. Visual inspection of the resulting volumes was performed by an echo-cardiography specialist, to verify apparent anatomical correctness, image quality and clinical value. We validated the geometrical accuracy of this volume compounding approach on two excised porcine mitral valve units, shown in Figure 5. These valves were imaged using a Philips Epiq system with an X8-2T TEE probe, with volumes being captured sequentially from a mid-esophageal point, along a 3D printed path simulating the esophagus to a transgastric position. The valve was also stained with iodine and imaged with a cone-beam CT scanner (Medtronic O-Arm, Medtronic Canada, Brampton, ON, Canada) to provide ground truth data. As shown in Figure 6, the ultrasound volumes were compounded using our described registration approach with the monogenic signal-based blending method. Linear measurements were then made of the visible chordae structures in both volumes.

3. Results

3.1. Porcine Model

The compounded volumes visually replicated the anatomical structures visible in the ground truth CT scan. As shown in Figure 7, the mitral valve leaflets, papillary muscles and individual chordae are clearly visible in the compounded volume. The compounded volume and CT are compared in

Table 1. Volume comparison metrics between compounded echo and CT.

Excised Valve	Chordae Measurement Absolute Difference (mm)
Valve 1	0.7 ± 0.6
Valve 2	0.6 ± 0.6

for each valve. For both volumes, four chordae that were easily visible in the compounded echo and CT were measured from the papillary muscle tip to the leaflet insertion point, and the average absolute difference between US and CT length measurements was computed. The measured chordae lengths ranged from 22.0 mm to 36.0 mm.

3.2. Patient Images

We processed image volumes acquired from two patients to create compounded volumes that were visually inspected by a cardiac anaesthesiologist specializing in echocardiography. The general consensus was that both volumes maintain acceptable clinical quality for the mitral valve leaflets and that the chordae tendineae were very clearly visible in the volumes for both patients. The compounding process enabled the contrast between background noise and tissue to be more evident. Overall, compounded volumes exhibit an improvement in image quality and include a wider field of view with little signal dropout, shown in Figure 8. The overall conclusion was that these volumes represented an improvement over existing techniques, both in image quality and range of structures visible.

4. Discussion

Spatial compounding has been demonstrated for many applications to improve field of view and image quality. Incorporating image information into the weighting function shows clear improvement over prior blending approaches. The combined distance and oriented symmetry weighting improves image quality and helps eliminate blending artefacts where the separate image acquisitions did not entirely agree. Compared to our previous results for this application, the registration strategy incorporating simultaneous group registration as the initial step improved robustness and eliminated the effect of initialization order, helping to prevent registration failure where the volumes do not reach alignment.

For the application of mitral valve procedure planning, we show that spatially compounded 3D echocardiography volumes are able to capture the complex structures in the left ventricle. Utilizing spatial compounding reduces image noise and provides a single volume containing the mitral valve, chordae tendineae, and papillary muscles, enabling clinicians to work from a single volume, instead of reconciling multiple separate volumes together. We demonstrate on porcine models that our spatial compounding method using a 3D TEE probe can reproduce the structures captured by a CT scan with high geometrical accuracy. Although the chordae appear thicker in the compounded volume, the separate individual chordae can still be identified from leaflet to papillary muscle. We found that the length of the chordae can be accurately measured from the compounded volume, as the thickening artefact does not affect the measurement of the length of the chordae.

The workflow described here can be integrated into the clinical standard of care, requiring only 4–5 acquisitions with approximately 80% overlap. Standard practice currently includes diagnostic 3D TEE for patients undergoing mitral valve procedures and transgastric images are already acquired as part of this process in the form of 2D long-axis views in an attempt to capture the chordae in their entirety. This compounding technique enables the transgastric images to be acquired as a series of 3D volumes instead of the traditional 2D views, while still maintaining visibility of the entire chordae structures. Our workflow makes it possible for clinicians to map almost the entire chordal structure in 3D from the leaflet to the anchoring point in the left ventricle, greatly improving the surgeon’s ability to optimize lengths for the introduced neochordae. Another instance where detailed, compounded 3D echocardiography has potential is the early diagnosis of endocarditis, where individual 3D image volume analysis can often remain ambiguous [16]. The volumes produced by applying 3D spatial compounding to TEE imaging capture the entire valve complex. Currently, many procedures require additional imaging in the form of cardiac CT/MRI to accurately perform diagnoses or plan for procedures [17]. Spatially compounded multi-view echo has the potential to add to the clinical imaging workflow by providing similar levels of information to cardiac CT at high frame rates, low cost and no radiation exposure to patients. Further validation of the clinical utility of this method will aim to demonstrate the effectiveness of the compounded echo, in particular related to cases in which cardiac CT is currently necessary.

Future work on this compounding method may include further improvements to registration speed and accuracy, as well as further exploration of blending approaches. Currently, the compounding process is performed offline due to computational requirements, with the process taking roughly 1 h for a five volume data set of two beat acquisitions. However, further optimizations may enable a real-time compounding approach where the final volume is created as the volumes are acquired, which may also allow for guidance to be provided to the operator to ensure the volumes are collected with sufficient overlap. Incorporating image-based real-time tracking algorithms and incorporating GPU acceleration may allow for active guidance of the volume acquisition relative to the initial position. Currently, this is a major drawback of the approach using offline processing, as in cases where the acquisitions have insufficient clarity or overlap, compounding will fail and the imaging session will have already concluded. With improvements to the compounding process, extensions to the tricuspid valve could also be possible, extending the range of clinical applications for this work. Additional validation of the compounding approach for mitral chordae on a larger cohort of patients will be performed to evaluate how accurately neochord sizes can be predicted from these extended volumes. Further work needs to be done to evaluate the effects of multi-view compounding on image quality using patient image data. Quantitative evaluation of image quality on a larger image set will be carried out by looking at the improvements in the contrast to noise ratio and image sharpness, as described in prior volume compounding work [5,6].

5. Conclusions

We describe a workflow for capturing a series of volumes using a TEE probe during standard diagnostic imaging that can then be registered and compounded together. Furthermore, we demonstrate improvements to the compounding process in registration robustness and final image quality. These compounded volumes capture the sub-valvular structures of interest for cardiac procedure planning. Capturing the necessary additional volumes can be performed while only adding an additional ten minutes to the time for the current standard-of-care diagnostic imaging protocol. We validate the geometrical accuracy of the compounding approach on two excised porcine valves, finding the measurement error between compounded ultrasound and ground-truth CT to be 0.7 ± 0.6 mm and 0.6 ± 0.6 mm, respectively. This method is able to provide clinicians with a single volume that captures the mitral valve and the sub-valvular structures, which will enhance the procedure planning process.