Applications of Machine Learning and Neural Networks for FT-ICR Mass Measurements with SIPT

Abstract

The single-ion Penning trap (SIPT) at the Low-Energy Beam Ion Trapping Facility has been developed to perform precision Penning trap mass measurements of single ions, ideal for the study of exotic nuclei available only at low rates at the Facility for Rare Isotope Beams (FRIB). Single-ion signals are very weak—especially if the ion is singly charged—and the few meaningful ion signals must be disentangled from an often larger noise background. A useful approach for simulating Fourier transform ion cyclotron resonance signals is outlined and shown to be equivalent to the established yet computationally intense method. Applications of supervised machine learning algorithms for classifying background signals are discussed, and their accuracies are shown to be ≈65% for the weakest signals of interest to SIPT. Additionally, a deep neural network capable of accurately predicting important characteristics of the ions observed by their image charge signal is discussed. Signal classification on an experimental noise dataset was shown to have a false-positive classification rate of 10.5%, and 3.5% following additional filtering. The application of the deep neural network to an experimental ⁸⁵Rb⁺ dataset is presented, suggesting that SIPT is sensitive to single-ion signals. Lastly, the implications for future experiments are discussed.

1. Introduction

Precision nuclear mass measurements are fundamental quantities needed in practically every area of nuclear physics research. The masses of rare isotopes far beyond the valley of stability are of particular importance for both nuclear structure and astrophysics calculations [1,2,3,4,5]. Such measurements can be used to extract a large range of nuclear structure information, including shell structure, nuclear deformation, pairing, and effective interactions, among others. New accelerator facilities, such as the Facility for Rare Isotope Beams (FRIB), are expected to improve rare isotope production by orders of magnitude; however, many isotopes very far from stability would only be delivered at rates below ∼${10}^{-5}$ ions/s [6]—roughly one ion per day. The Low-Energy Beam Ion Trapping (LEBIT) facility at FRIB performs precision mass measurements of these isotopes using Penning trap mass spectrometry—the most precise method known to date [7,8]. Techniques such as time-of-flight ion cyclotron resonance (TOF-ICR) [9] and phase-imaging ion cyclotron resonance (PI-ICR) [10], while flexible, are destructive and require hundreds of ions to complete a mass measurement. Measuring mass information to the needed precision for shell structure evolution [11,12] away from stability is difficult and often infeasible given the low delivery rates of the isotopes of interest, particularly near the doubly magic ¹⁰⁰Sn and ⁷⁸Ni which are of current interest.

The single-ion Penning trap (SIPT) developed by LEBIT addresses these shortcomings with the narrowband Fourier transform ion cyclotron resonance (FT-ICR) technique [13]. SIPT has been developed specifically to achieve the needed mass precision for single isotopes, which are of high importance to nuclear structure and astrophysics studies. This technique is suited to low delivery rates, as a measurement relies only on recording the image current produced by ions in the trap. This is of particular use for the most exotic rare isotopes, such as those near the proton/neutron drip lines. FT-ICR measurements can be performed with only a single ion given a suitably low-noise amplification circuit. The challenge in performing these measurements is that the weak ion signals and electronic noise are often comparable and can be easily conflated. Various measured and simulated cases are presented in Figure 1, illustrating the difficulty associated with this problem.

The classification of these signals as either background or from ions is a problem well suited to supervised machine learning techniques [14,15]. Caution is often required when utilizing machine learning as the models are limited in parameter scope and are susceptible to over-fitting a trend to data. As such, stable numerical approaches should always be favored when possible. This is not easily feasible for the problem at hand: cases where ion signals and noise have comparable amplitudes become exceptionally difficult to ‘fit’, if not impossible. Machine learning is applied as a means of extracting as much information from each signal as possible, with the understanding that predictions are probabilistic.

Machine learning algorithms use a large set of labeled data to predict the most likely classification for an unknown instance. Machine learning, including deep neural networks, has recently been applied to a variety of nuclear physics problems such as determining nuclear mass [16,17], nuclear charge radius [18], and ground-state energy [19], among many others. Applications of machine learning for FT-ICR mass measurements have been explored, but tend to focus on sample analysis such as biomarker profiling [20,21], accounting for space charge effects [22,23], or molecular mass spectrometry [24]. These applications are of no particular significance to precision isotope mass measurements with SIPT. No previous work could be identified addressing the classification of background signals using machine learning on FT-ICR data, or for determining ion characteristics in an FT-ICR Penning trap.

In this paper, several machine learning models are presented which are used to identify ion signals from electronic noise and extract important characteristics of ions within the detector. The necessary background regarding the mass measurement technique, theoretical models, and the specifics of the SIPT detector is presented in Section 2. A method for efficiently simulating realistic SIPT FT-ICR signals is discussed in Section 3. Section 4 highlights the procedure for applying supervised machine learning algorithms for signal vs. noise classification. In Section 5, a deep neural network (DNN) predicts ion characteristics for a large set of measured isotope signals in SIPT. Section 6 applies the classification algorithms to an experimental dataset and estimates the characteristics of the ions using the neural network. Conclusions and directions for future work are provided in Section 7.

2. Background

2.1. Ion Motion in a Penning Trap

Using Penning traps for ion capture and excitation is well established [8], and a brief summary is presented here for context. A Penning trap uses a homogeneous magnetic field and a quadrupolar electrostatic field which confines charged particles in space. Mass measurements are made possible through the determination of the characteristic cyclotron frequency of a trapped ion. This frequency is related to the ion’s mass by

$${\omega }_c=2\pi {\nu }_c=\frac{qB}{m}$$

(1)

where q and m are the ion charge and mass, respectively, and B is the magnetic field strength. This ion motion is a superposition of an axial motion (defined as the z-direction and parallel to the magnetic field) with frequency

$${\omega }_z=2\pi {\nu }_z=\sqrt{\frac{q{U}_0}{m{d}^2}}$$

(2)

and two radial motions with frequencies

$${\omega }_{\pm }=2\pi {\nu }_{\pm }=\frac{{\omega }_c}{2}\pm \sqrt{\frac{{\omega }_c^2}{4}-\frac{{\omega }_z^2}{2}}$$

(3)

where $U_0$ is the potential difference between the endcap and ring electrodes, and d is a characteristic trap parameter. Furthermore, the eigenfrequencies satisfy the following relationship:

$$\begin{array}{cc}\hfill {\omega }_c=& {\omega }_{+}+{\omega }_{-}.\hfill \end{array}$$

(4)

The position of an ion in the trap is described by the following equations:

$$\begin{array}{cc}\hfill x=& {\rho }_{+}cos({\omega }_{+}t+{\varphi }_{+,0})+{\rho }_{-}cos({\omega }_{-}t+{\varphi }_{-,0})\hfill \\ \hfill y=& {\rho }_{+}sin({\omega }_{+}t+{\varphi }_{+,0})+{\rho }_{-}sin({\omega }_{-}t+{\varphi }_{-,0})\hfill \\ \hfill z=& {\rho }_{z}cos({\omega }_{z}t+{\varphi }_{z,0})\hfill \end{array}$$

(5)

where ${\rho }_{\pm }$, ${\varphi }_{\pm ,0}$ and ${\rho }_z$, ${\varphi }_{z,0}$ are the radial amplitudes and initial phases of the radial and axial eigenmotions. Dipole and quadrupole radio frequency (RF) excitations can be applied to the Penning trap to modify and convert between ${\rho }_{+}$ and ${\rho }_{-}$; further specifics are discussed in [9]. The application of these excitations also impacts the eigenmotion phases, ${\varphi }_{\pm ,0}$, and a derivation of this effect is presented in [25]. Notably, for multiple ions in the trap, longer RF dipole excitations of ${\rho }_{\pm }$ correspond to a tightening of the distribution of ${\varphi }_{\pm }$ such that the relative ion spatial distribution remains constant regardless of excitation time. The effect of a sufficiently long dipole excitation is observed by the center of the ion ‘spot’ moving to a larger radius while the shape of the spot remains unchanged.

2.2. Fourier Transform Ion Cyclotron Resonance Technique

The non-destructive narrowband Fourier transform ion cyclotron resonance (FT-ICR) technique used by SIPT is sensitive to the image current on the Penning trap ring electrode segments, produced as ions undergo their characteristic eigenmotions [13]. A narrowband approach limits the observable frequency range to be near that of the eigenmotion being probed for a given isotope [26]. The resulting image current signal is a combination of noise as well as contributions from each ion present in the trap. This signal is very weak—typically of the order of femtoamperes—and requires further amplification to be readable. SIPT uses a low-noise, high-quality factor resonator circuit sufficient for detecting the image charge of a singly charged ion. Both the SIPT and detection circuit are operated at cryogenic temperatures to further reduce thermal noise, improving the signal-to-noise ratio of the detected ion signals by a factor of eight compared to room temperatures. A discrete Fourier transform (DFT) [27] is then applied to the time-domain signal to produce the frequency spectrum, from which the eigenfrequency of interest can be identified. Often, this is achieved by fitting a peak in the frequency spectrum to a characteristic Lorentzian curve of the form

$$L\left(\nu \right)=L_0+\frac{\left|b\right|·c}{{(\nu -{\nu }_0)}^2+c^2}$$

(6)

where $L_0$ is an initial offset, b and c are the characteristic height and width, respectively, and ${\nu }_0$ is the distribution center. In this formulation, the total area of the distribution is $b\pi$. Since the eigenfrequency is expected at the peak value, ${\nu }_0$, one can determine the isotope’s mass from Equations (1) and (4).

The ion image current signal is directly proportional to the ion’s charge: FT-ICR is increasingly more sensitive for multiply charged ions [13]. This poses an even greater challenge for measurements of a single singly charged ion. Given the negative impact of noise on the potential resolving power, the ability to identify, isolate, and filter noise is highly desirable. This problem has been the subject of various studies [28,29,30,31], and various FT-ICR techniques have been developed to improve the signal-to-noise ratio achievable [32,33,34]. However, these techniques are not suited to the narrowband FT-ICR technique, or have limited applicability for precision mass measurements of rare or exotic nuclei. Furthermore, selective noise filtering techniques are not effective for the typically weak SIPT ion signals as they can be significantly distorted.

The FT-ICR signal strength for a given eigenmotion(s) is also highly dependent on which of the segmented ring electrodes in the Penning trap are used for detection or excitation [35,36]. Often this detection scheme is chosen to maximize the signal amplitude of the eigenfrequency being probed. Simulations were performed in [36] to determine an optimal configuration of detection/excitation electrodes for cyclotron and reduced cyclotron frequency pickup. In [35], simultaneous detection of ${\omega }_c$ and ${\omega }_{+}$ using a single detection circuit was demonstrated, which provides an unambiguous ion signature if both frequencies are observed. However, this method is still in development for SIPT and would also require frequency identification in two separate regions. For these reasons, the measurements discussed in this work consider the established dipole pickup of only the modified cyclotron frequency ${\nu }_{+}$, whose scheme is illustrated in Figure 2.

Ion motion in the Penning trap also plays an important role in the achievable signal strengths. As discussed in Section 2.1, the phase distribution of ions following a dipole RF excitation converges for longer excitation times. This is of critical importance for FT-ICR measurements as each signal is a superposition of all ion image current signals. As such, for multiple ions, destructive interference of the ion image current signals can occur, even more so if the phase distribution spans a large range. A large phase distribution is not always avoidable, and is limited in part by the injection of ions into the Penning trap: reduced cyclotron motion is introduced as ions cross magnetic field lines due to imperfect alignment. Longer duration dipole RF excitations cannot always be used to reduce the phase spread as electric field imperfections are amplified at larger radii [9]. An additional consideration is the impact of axial motion introduced by non-optimal ion injection into the Penning trap. Realistic simulations of FT-ICR signals, such as the one presented in Section 3.3, suggest that axial oscillations in SIPT have a negligible impact (∼ 1 part in ${10}^5$ for ${\rho }_z\le 0.25{\rho }_{+}$) on the eigenfrequency signal strength, and as such are not considered further for this work.

3. Numerical Simulation

3.1. Noise Floor Determination

Considering a realistic construction of the SIPT detector noise floor, the dominant source of noise in FT-ICR signals is expected to be root mean-squared (RMS) Johnson–Nyquist noise [37]. Assuming an ideal resistor, RMS noise is approximately white [38]. The broadband noise power spectral density (PSD) of an FT-ICR detection circuit, however, is generally not white, since it is subject to resonances from the RLC amplifier and other external sources. The narrowband FT-ICR measurements used by SIPT focus on a small frequency band around the eigenfrequency being probed. Within a sufficiently narrow band, noise can be reasonably approximated as white. A model for the simulated noise signal at any point in time is defined simply as

$$S_{\mathrm{noise}}\left(t\right)=U(0,A_{\mathrm{noise}})$$

(7)

where U denotes the uniform random distribution from zero to a maximum of $A_{\mathrm{noise}}$. Measured background signals from SIPT can then be compared to a simulated noise signal of the same length by their power spectral densities (PSDs). When determining the PSD, a spectral bandwidth of 100 Hz centered about the expected ion frequency was used, as it captures the maximum expected width of the ion signals by at least a factor of ten. Comparisons of the average PSDs for a large set of both simulated and measured noise signals are shown in Figure 3. A reasonable white noise approximation with an amplitude of $A_{\mathrm{noise}}\approx 4.85$ mV was determined to best fit the measured background noise power from SIPT.

3.2. Simulated Signal-to-Noise Ratio

The signal-to-noise ratio (SNR) is a useful metric for quantifying the strength of a desired signal to that of the intrinsic noise. The ability to accurately identify weak signals is directly related to this ratio: an ion signal with a larger SNR becomes more clearly defined from noise. A general definition for the signal-to-noise ratio is

$$\mathrm{SNR}=\frac{P_{\mathrm{signal}}}{P_{\mathrm{noise}}}$$

(8)

where $P_{\mathrm{signal}, \mathrm{noise}}$ are the powers of the signal and noise, respectively. The power for either the signal or noise is calculated as the integrated Fourier spectrum over a finite spectral bandwidth. In the case of simulated data, the SNR can be determined in this manner since both the noise and ion signal are known independently.

For an FT-ICR Penning trap, assuming RMS Johnson–Nyquist noise, the signal-to-noise ratio (SNR) is approximated by

$$\mathrm{SNR}=\kappa \frac{\sqrt{\pi }}{4}Nqe\left(\frac{\rho }{{\rho }_0}\right)\sqrt{\frac{\nu }{\Delta \nu }}\sqrt{\frac{Q}{k_{B}TC}}$$

(9)

where $\kappa \approx 0.93$ is a geometrical parameter that accounts for the fact that the pickup electrodes are not infinite flat planes; N is the number of ions; $qe$ is the ion charge; $\rho /{\rho }_0$ is the radius of the ions’ orbit relative to the trap radius; $\nu /\Delta \nu$ is the ratio of the ion eigenfrequency to the spectral bandwidth; $k_B$ is the Boltzmann constant; and Q, T, and C are the quality factor, temperature, and capacitance of the detection circuit, respectively [7,39]. Maximizing the quality factor of the RLC resonator is essential to achieving reasonable SNR values for single-ion signals, and a baseline of $Q\sim 1000$ is necessary. For the SIPT, typical values for these parameters are $\rho /{\rho }_0\approx 0.5$, $Q=2785$, $T=5$ K, and $C\approx 30$ pF. The capacitance of the resonator is tuned for a given isotope such that the SIPT resonator peaks at the eigenfrequency being measured. Lastly, the radius of the ion in the trap is dependent upon the duration of the initial dipole excitation of the isotope eigenmotion. For single ions of ¹⁰⁰Sn⁺ and ⁷⁸Ni⁺, SIPT is expected to achieve SNR’s of roughly $5.5/\sqrt{\Delta \nu } {\mathrm{Hz}}^{1/2}$. The spectral bandwidth is left undetermined as it is limited by the measurement sample rate and acquisition time. A more precise measurement will yield a narrower resonance for the eigenfrequency being probed, which improves the SNR as a narrower spectral bandwidth can be used.

3.3. Methods for Simulating FT-ICR Signals

A rigorous method to simulate the image current of an ion undergoing cyclotron motion for a given Penning trap ring electrode configuration has been previously established [9], and a brief summary is presented here for completeness. The voltage readout of the ion image charge current on the Penning trap ring electrodes is given by

$$S\left(t\right)=R_{\mathrm{eff}}q\overrightarrow{v}\left(t\right)·{\overrightarrow{E}}_1$$

(10)

where $R_{\mathrm{eff}}$ is the effective resistance of the detector circuit, which is dependent on the intrinsic resonator frequency; q and $\overrightarrow{v}\left(t\right)$ are the ion charge and velocity, respectively; and ${\overrightarrow{E}}_1=\overrightarrow{E}/V$ is the unit electric field generated by a $\pm 1$ V potential difference applied to the detection electrodes. A discrete approximation of the electric field can be computed using SIMION [40] for the desired pickup scheme of the Penning trap ring electrodes. The ion velocity is found easily by differentiating the equations for ion position in Equation (5).

However, this method is computationally intensive as the fields must be determined for the ion at each new position. This becomes problematic for the machine learning algorithms used later in this work as they rely on a large dataset of these signals to adequately encompass the problem complexity. For example, the neural network in Section 5 requires the simulation of ∼${10}^8$ unique ion signals to achieve an acceptable accuracy. It was estimated that approximately 200 days of calculation time using 64 fully dedicated CPUs would be required to complete all the required simulations on a state-of-the-art high-performance computing farm. Clearly, a faster simulation method is highly desirable. Given the expected periodic behavior of the signals, a simplified construction is used, where each ion’s image current is sinusoidal and acts independently. The overall voltage signal from all ions is simply a superposition of each ion’s contribution:

$$S\left(t\right)=S_{\mathrm{noise}}\left(t\right)+\sum _{i=1}^N{A}_{i}sin\left(2\pi t{f}_i+{\varphi }_i\right)$$

(11)

where N is the number of ions composing the signal; $S_{\mathrm{noise}}$ is the signal contribution from noise; and $A_i$, $f_i$, and ${\varphi }_i$ are the amplitude, frequency, and initial phase, respectively, of the $i\mathrm{th}$ ion. This approach lends itself well to vectorization and is, thus, significantly faster (approximately 50 times faster than the rigorous method).

The general sum-of-sines algorithm is detailed in Algorithm 1. Each ion has its eigenfrequency selected from a normal distribution with a standard deviation of $0.5$ Hz. This well encompasses the observed experimental uncertainty. Similarly, the voltage signal amplitude for a given ion is also selected from a normal distribution with a typical standard deviation of $0.005$ mV. This approximates the beam spot radial distribution. The noise contribution to the signal is modeled by Equation (7) with $A_{\mathrm{noise}}=4.85$ mV. Lastly, the phase spread, ${\varphi }_i$, discussed in Section 2.1, is also chosen from a uniformly random distribution with a maximum value of ${\varphi }_{0,\max }$. Each simulated time-domain signal is sampled over 24,000 points, each 5 μs apart, for a total duration of 120 ms. The simulated frequency-domain signal was then computed using a fast Fourier transform (FFT).   

Algorithm 1: Serial method for simulating an FT-ICR time-domain signal using the ‘sum of sines’ approach. The inputs are as follows: $N_{\mathrm{ions}}$ is the number of ions in the Penning trap; f and ${\sigma }_f$ define the ion eigenfrequency normal distribution; A and ${\sigma }_A$ define the single-ion amplitude normal distribution; ${\varphi }_{0,\max }$ is the maximum eigenmotion phase; $A_{\mathrm{noise}}$ is the noise amplitude; $\Delta t$, $N_{\mathrm{samples}}$ are the sample time and number of samples, respectively. The method UniformRandom (low, high, N = 1) samples N instances from a uniformly random distribution within the range defined from low (inclusive) to high (exclusive), and NormalRandom ($\mu , \sigma$) randomly samples a Gaussian/normal distribution defined by a mean value $\mu$ and standard deviation $\sigma$.

To compare the two methods and ensure the proposed method adequately approximates the more realistic SIMION approach, the SNR distributions for different numbers of ions in the Penning trap are analyzed for both approaches. Again, a spectral bandwidth of 100 Hz about the known eigenfrequency is used, capturing the width of expected ion signals by at least a factor of ten. The noise level of $A_{\mathrm{noise}}=4.85$ mV determined in Section 3.1 is used. The relationship between the SNR and the ion position in the trap, $\rho /{\rho }_0$ for the rigorous method, and SNR to the single-ion signal amplitude, $A_{\mathrm{single}}$ for the sum-of-sines approach, were determined at a 95% confidence level as

$$\begin{array}{cc}\hfill \mathrm{SNR}= & 0.229673\left(13\right)\frac{\rho }{{\rho }_0}\hfill \end{array}$$

(12)

$$\begin{array}{cc}\hfill \mathrm{SNR}= & 9.896849\left(3\right)\left[{\mathrm{mV}}^{-1}\right]A_{\mathrm{single}}\hfill \end{array}$$

(13)

where both have offsets compatible with zero, and the SNR was determined over a 100 Hz spectral bandwidth. Using the expected SNR’s for ¹⁰⁰Sn⁺ and ⁷⁸Ni⁺, the approximate single-ion signal amplitude is $A_{\mathrm{single}}$∼ 0.02 mV. This is used as a point of reference when determining a reasonable amplitude for ⁸⁵Rb, for which a large dataset of signals have been collected. Furthermore, combining these relationships provides a correlation between the relative ion radius in the Penning trap to the simulated amplitude for a single ion:

$$\frac{\rho }{{\rho }_0}=43.063\left(2\right)\left[{\mathrm{mV}}^{-1}\right]·A_{\mathrm{single}}$$

(14)

where the uncertainty is determined in quadrature at a 95% confidence level. A comparison of the signals generated from Equations (10) and (11) for equivalent SNR’s using Equation (14) and different phase spreads is presented in Figure 4.

Of primary interest is the relative positioning and widths of the distributions. The results are equivalent, so the approximation presented in Equation (11) is used for the remaining work.

4. Identification of Ion Signals

A reliable means of identifying ion signals, especially in cases with small signal-to-noise ratios, is fundamental to performing the mass measurement. Machine learning algorithms are well suited to this type of task. These algorithms (also known as models) require two sets of information: the ‘feature set’ (also known as model input) and a ‘prediction set’ (also known as model output). The feature set is the data provided to the algorithm upon which predictions are based on. A prediction set is the matching pair to the input that informs the algorithm of the true output. When training a machine learning model, a large dataset of known inputs and true outputs is used to optimize its algorithm in some fashion. The performance of each classifier is evaluated using its precision: the number of true-positive classifications relative to the true number of true-positive and false-positive instances. Precision was chosen as the cost of a false-positive classification (noise mistaken for an ion signal) is deemed greater than the cost of a false-negative classification (ion signal mistaken for noise) [41]. A standard means of evaluating the performance of a classifier is using cross-validation [42]. Specifically, k-fold cross-validation is used throughout this work to compare the precision between models.

For identification of ion signals, the model used is a ‘binary classifier’ which labels a given feature set as either noise or an ion signal. Four classifiers from the Scikit-learn library [43] are considered: k-nearest neighbors (kNN), Gaussian naive Bayes (GNB), support vector machine (SVM), and logistic regression (LR). These algorithms were chosen based on their performance and suitability for the frequency data used to make the classification. There are many other standard algorithms, like decision trees, linear regression, and neural networks. However, these methods showed no particular benefits, and initial investigations showed no performance increase over other models. These options were not considered further.

4.1. Dataset Construction for Signal Classifiers

The frequency spectra, relative to the expected ion frequency, of the SIPT signals are used for the classifier feature set. In principle, these should be independent of the frequency magnitude of the signal, allowing the classifier to be used for signals generated by ions of any mass. This assumes the noise remains white at any frequency magnitude, an assumption which is largely preserved for a narrow frequency band. The classifiers are dependent of the frequency resolution of the signals: this is limited by the sampling rate and total acquisition time of the signal. Signals with higher resolution, however, can easily be down-sampled to match the required frequency resolution of the classifier. The parallel nature of the signal model of Equation (11) allows for new training datasets to be quickly generated if needed. The work presented here considers time-domain signals with a total of 24,000 samples, each 5 μs apart, for a total time of 120 ms.

Including noise data at frequencies far beyond the expected ion frequency has the potential of ‘confusing’ classifiers, as such datapoints are ‘informationless’: a known concern for classifier reliability and efficiency [44,45]. Furthermore, the frequency resolution of the FFT allows signal peaks to be either clarified or obfuscated within the surrounding noise. Additional frequency granularity can be interpolated from an FFT by padding the time-domain signal with additional zeros. This process is referred to as zero-padding [46]. A zero-padding multiplier, $\alpha$, was defined to mean appending $\alpha N_t$ zeros to the time-domain signal of length $N_t$. Beyond aiding classification, including further interpolated frequency information is also important for accurately fitting the spectrum to the characteristic Lorentzian curve. Minimal improvement was observed in classification performance beyond a padding multiplier of one to both the left and right ends of the signal. However, a minimum multiplier of two was observed to significantly improve the reliability of curve fitting.

Since the narrowband FT-ICR approach is used, the FFT spectrum is limited to a narrow range encompassing the expected ion eigenfrequency. A large frequency range is potentially important for generalizing noise trends, but this can also overshadow the importance of a valid ion peak. The optimal frequency range should then maximize the classifier performance while minimizing false-positive and false-negative classifications. An investigation of classification precision using a 10-fold cross-validation [47] approach with 10,000 samples, for different frequency ranges centered about ${\nu }_c$, is shown in Figure 5.

One pertinent feature is the stability of the classifiers across frequency ranges. A higher degree of stability suggests a robust determination of the relevant frequency information regardless of additional noise. All the classifiers display some level of instability for narrower frequency windows. Another important aspect to consider is that smaller frequency ranges increased the rate of false-positive classifications. This is likely due to an over-weighting of features near the expected eigenfrequency.

Taking these findings into consideration, the following method was devised. First, the time-domain signal for each simulated measurement was determined. After which, a zero-padding of twice the length the time-domain set was appended to both sides. Then, a fast Fourier transform was performed on the padded signal. Lastly, the frequency spectrum was reduced to only include frequencies within a 25 Hz window, centered at the expected eigenfrequency. In total, a set of 20,000 signals were generated for training and testing the accuracies of the classifiers.

4.2. Optimal Classifier Hyperparameters

For the classifiers of interest, choosing appropriate model hyperparameters is necessary to efficiently and accurately predict an unseen input. To accomplish this task in a systematic manner, a Bayesian optimization [48] approach is used. Specifically, the BayesSearchCV in the Python scikit-optimize [49] library is used for the primary hyperparameters for each classifier. A set of 10,000 simulated Fourier domain signals with an equal sampling of background and single-ion signals was used to train each model using a 5-fold cross-validation technique [47]. Initial ranges for hyperparameters in the optimization were chosen using a coarse grid over a large range of values and moving to a finer grid where clear positive trends were observed. In the case where multiple parameter sets were equivalently ranked, the set with the fastest training and prediction was chosen. A summary of the results for each classifier is presented in

Table 1. Summary of the optimal model hyperparameters found using a Bayesian optimization for four classifiers: k-nearest neighbors (kNN), Gaussian naive Bayes (GNB), support vector machine (SVM), and logistic regression (LR). Classifier precision was determined from a cross-validation approach with 10,000 samples. The signals with ions present were chosen to have $\mathrm{SNR}=0.23$ normally distributed with an uncertainty of ${\sigma }_{\mathrm{SNR}}=0.05$.

Classifier	Parameter	Value	Precision
kNN	Algorithm	Ball Tree	0.711
	Leaf Size	46
	k Neighbors	35
GNB	Variable Smoothing	0	0.704
SVM	C	100	0.715
	$\gamma$	1.895
	Degree	8
	Kernel	rbf
LR	C	39	0.735
	Solver	Saga
	Penalty	ℓ2

.

4.3. Classification of Ion Signals

A concern for experimental applications of these classifiers are mis-classifications, wherein noise propagates into the final mass calculation, or valid ion data are incorrectly identified as noise. This can be mitigated, in part, by a priori filtering of the signals. The simplest scheme would be to simply filter signals above a set SNR threshold which meets an acceptable false-positive classification rate. However, this is infeasible/inefficient for most single-ion measurements since they have signal amplitudes which can be on the order of the noise itself. A more intelligent scheme can be accomplished by first fitting a Lorentzian of Equation (6) to the frequency spectrum. Initial parameter values for this optimization are selected under the assumption of a valid ion signal, i.e., ${\nu }_0={\nu }_{c,\pm }$. The filtering scheme uses a priori knowledge of the ion being measured such that restrictions can be placed on the Lorentzian fit parameters. For example, requiring the fitted center frequency to be within a finite range of the expected cyclotron frequency, requiring the characteristic Lorentzian width and height to be within reasonable bounds, and requiring the fit ${\mathrm{R}}^2$ to meet a threshold. The filtering restrictions can be set to arbitrary limits at the potential expense of rejecting additional ion signals.

To classify a given signal, all four classifiers independently predict and at least two positive ‘votes’ are required for a signal to be classified as containing ion information. Following this, a Lorentzian distribution is fit to each signal classified as containing ion information and the filtering scheme described above is applied. Signals which meet all the filtering restrictions are determined to contain meaningful ion information. As one might expect, classification performance is directly related to the SNR of a given signal, where lower values become increasingly difficult to differentiate from noise. A study of the expected classification precision on single-ion signals for different SNR ratios is presented in Figure 6.

After $\mathrm{SNR}\approx 1.5$, the classifiers are able to parse noise from signals with exceptionally high precision. However, SIPT is most concerned with measurements of signals below this level, where classification precision drops steeply. For example, ¹⁰⁰Sn⁺ is expected to have a precision of about 60–70% for single ions.

It is critical that the classification of these signals does not impact the extracted frequency (and consequentially the measured mass). This may be most easily found in the classification of noise signals as those produced by an ion, whose fitted frequency peak introduces an additional uncertainty into the final result. Furthermore, the superposition of the noise on top of the ion signal has the potential to introduce an additional statistical uncertainty. This problem is two-fold: for small signal-to-noise ratios, the effect is amplified and the classifier accuracy is diminished. The case for signals with SNRs expected for ¹⁰⁰Sn⁺ was studied: simulations of 1000 background and single-ion signals each were classified, and the frequency extracted from the fitted Lorentzian distribution. The average recovered frequency agreed with the simulation input within the prediction uncertainty. The potential for adding additional systematic and statistical uncertainties remains a topic for further investigation when performing high-precision measurements.

5. Extracting Ion Characteristics with Neural Networks

To better quantify SIPT’s functionality and experimental applicability, additional information about ions in the detector’s Penning trap is needed. Properties such as the number of ions producing a signal, the signal strength of each ion, and the phase spread of ion motion provide information on how SIPT can be optimized during an experiment to make the best use of the limited beam time. Unfortunately, a single SIPT signal is insufficient to determine these values with any confidence. For instance, signal-to-noise does not uniquely determine ion number due to a significant overlap in the signal-to-noise distributions, evident in Figure 4. Additionally, the amount of overlap in these SNR distributions is dependent on each ion’s charge and location in the trap (which determine its signal amplitude $A_i$) as well as the range of the reduced cyclotron motion phase of each ion. However, by considering a large set of signals whose properties of interest are similar, general trends emerge which are unique to the given characteristics. In this section, deep neural networks (DNNs) are utilized to predict these characteristics.

5.1. Construction of Datasets for Predicting Ion Characteristics

Preparing the training dataset for the DNN is a multi-step process. First, a set of signals with similar properties are assembled, uniquely characterizing the set. A sufficiently large sampling (∼${10}^4$) of individual simulated signals is produced where each has the same single-ion amplitude distribution, phase spread, and voltage offset. The assumption of constant eigenmotion phase spread and single-ion amplitude is consistent with how experiments are conducted, as the isotope and excitation radius are kept constant. The number of ions delivered from an experimental beam which create a given SIPT signal is assumed to follow a Poisson distribution. After generating the signal dataset, the classification scheme described in Section 4.3 is used to reject signals classified as only electronic background. A Lorentz distribution is then fit to the FFT spectra for each signal in the remaining set. Lastly, the filtering scheme described in Section 4.1 is then applied to remove any signal that does not meet all the criteria.

Following the signal generation and subsequent fitting, signals are characterized in a form suitable for predictions to be made by the DNN. While the SNR may seem like a natural choice, there are limitations with experimental signals: only an approximate SNR can be calculated and it varies widely based on the assumed noise spectrum. A defining feature of the FFT spectrum for an ion signal is a peak near the expected frequency, which is captured by the Lorentzian curve. The area of this fitted distribution was determined to well characterize a given signal, as a strong linear relationship between the noiseless signal area and the fitted Lorentzian area of a noisy signal was observed. This quantity is closely related to the SNR, but exhibits far more accuracy. As such, the sampling of signals is well characterized as a whole by a histogram of the signals’ fitted Lorentzian areas. This histogram is given sufficient granularity over a range of values such that all signals are represented and important trends are resolvable. At this point, a single training instance has been created: the histogram of Lorentzian areas is the model input, and the list of ion characteristics used to generate the signals is the model output.

The full dataset for the neural network is built by repeating the above process for a total of 130,000 instances. Each instance is generated using a unique combination of the Poisson mean, single-ion amplitude distribution, voltage offset, and phase spread. The size of this dataset is chosen to adequately reflect the problem’s scope. This dataset is then split into three sets: 96,000 are used for training the DNN, 24,000 for validation of the model during training, and 10,000 for evaluating the model after training.

5.2. Network Architecture and Training

Preliminary studies indicated that a simple feed-forward multi-layer neural network [50] provided equivalent if not superior predictive power to more complicated network structures. Additionally, this has the advantage of a smaller number of hyperparameters which simplifies network optimization. The DNN described here is implemented with Tensorflow Keras [51]. The model input is the histogram with 400 bins of fitted and filtered Lorentzian areas for each signal, as described above. The internal structure of the network contains three hidden layers with 500, 250, and 125 neurons. Each hidden layer uses rectified linear unit (ReLU) [52] activation with a dropout rate [53] of fifty percent. The network output contains the following five values: predicted Poisson mean ${\mu }_p$, single-ion signal amplitude $A_{\mathrm{single}}$ and its experimental standard deviation $\sigma \left(A_{\mathrm{single}}\right)$, a constant signal voltage offset $V_{\mathrm{offset}}$, and the eigenmotion phase spread outer limit, ${\varphi }_{0,\max }$. The neural network was trained with the Adam optimizer [54] set to a constant learning rate of ${10}^{-3}$. The network parameters (i.e., ‘neurons’) were optimized with a standard mean-squared error (MSE) loss function [55]. The model was then trained over 50 epochs with a batch size of 64, which showed a consistent flattening of the loss curve while avoiding significant over-fitting, evidenced by the validation dataset.

5.3. Model Uncertainty and Sensitivity Analysis

For a neural network, the complexity of the model is often reflected in the uncertainties associated with each output value. Uncertainty analysis is performed to quantify this effect. Additionally, a network may also experience sensitivity, where small changes to some input values result in drastic changes to the output. Determining the dependence of the outputs on the input parameters is referred to here as sensitivity analysis.

An analysis of the model uncertainty is performed using the MC Dropout [56] method. MC Dropout allows each dropout layer in the network model to continue excluding nodes when making a prediction; dropout would normally be disabled when a consistent prediction is desired. After a number of iterations, the resulting distribution for each output value can be analyzed to determine an uncertainty. Here, the standard deviation is used to determine the uncertainty. In

Table 2. A summary of the true and predicted parameters from the neural network for six unique simulated datasets. All predicted values are determined to 1 σ uncertainty, calculated using the MC Dropout method [56] for 100 iterations. Summaries for each output parameter can be found in Section 5.2.

	Sim. Dataset 0		Sim. Dataset 1		Sim. Dataset 2
	True	Prediction	True	Prediction	True	Prediction
μp	0.54	0.54(8)	0.37	0.43(8)	0.14	0.18(6)
Asingle (mV)	0.053	0.053(5)	0.051	0.051(3)	0.046	0.044(2)
σ(Asingle) (mV)	0.000835	0.0017(9)	0.00174	0.0023(8)	0.000357	0.0024(7)
Voffset (μV)	8.89	8.1(1.1)	9.79	8.4(1.0)	1.85	0.9(8)
${\varphi }_{0,\max }$ (rad)	4.85	4(1)	1.91	1.8(1.0)	4.47	3.8(9)
	Sim. Dataset 3		Sim. Dataset 4		Sim. Dataset 5
	True	Prediction	True	Prediction	True	Prediction
μp	0.36	0.37(7)	0.12	0.18(7)	0.29	0.34(8)
Asingle (mV)	0.027	0.029(3)	0.054	0.049(3)	0.052	0.053(5)
σ(Asingle) (mV)	0.000404	0.000264(12)	0.000377	0.003(6)	0.000078	0.00015(10)
Voffset (μV)	8.66	7.7(9)	4.45	4.0(5)	9.73	8.8(1.1)
${\varphi }_{0,\max }$ (rad)	3.8	4.1(7)	3.44	3.2(9)	5.53	4.2(1.1)

, model predictions are presented for six unique simulated datasets, chosen to be representative of various observed cases. In each case, uncertainties are determined via MC Dropout to one sigma. A comparison of the predicted outputs to the known values indicates that the network is able to accurately predict values within the uncertainty.

There are many methods for performing a sensitivity analysis on a neural network model, such as those presented in [57]. The most common techniques assume a finite dataset with no underlying model (i.e., no means of simulating data). A simplified approach is taken given the ability to easily simulate additional datasets. Recall that the model is trained based on simulated data where the output corresponds to a set of parameters which were used to generate the input. The sensitivity of the model is quantified by comparing a given input A to an input B which is simulated from the output of A. The magnitude of the similarity/difference between inputs A and B reveals the dependence of the network’s input and output. When performing this inverse calculation, a sufficiently large sampling of SIPT signals must be used to generate input B to minimize statistical fluctuations introduced by the stochastic simulations. Using ∼$80,000$ signals to construct input B was observed to reduce these fluctuations to less than 5% on average. In Figure 7, stability analyses for the six cases in Table 2 are presented. Close agreement, i.e., small residuals, between the histograms in all cases suggest minimal sensitivity.

6. Applications to Experimental Data

The DNN-based mechanism described above is applied to an experimental dataset obtained with ⁸⁵Rb⁺ ions produced by the LEBIT offline test ion sources. A brief description of the dataset is presented here; further details regarding the LEBIT experimental facility and methods for collecting these data can be found in [36]. Before injecting ions into the SIPT, a 0.25 mm collimator placed within the drift tube before the trap was used to minimize spread in the ions’ radii. The time-domain signal was collected at 24,000 points, each 5 μs apart, for a total of 120 ms. A total of 33,297 signals were collected while the rubidium source was active, and an additional 18,000 signals were collected while no ions were allowed into the trap. The ion source was operated at low rates such that the majority of ion signals were likely from single ions. Following the experiment, and based on the findings of Section 4.1, each time-domain signal was transformed to the frequency domain using an FFT; a zero-padding of twice the length of the time-domain set was appended to both sides before taking the FFT. The frequency data were then truncated to $\pm 25$ Hz of the expected eigenfrequency.

6.1. Classification of Experimental Signals

All four classifiers described in Section 4, set with the hyperparameters described in Table 1, were trained on 10k simulated signals: 5k background, 3k single-ion, 1k two-ion, and 500 three- and four-ion signals each. Majority voting between all four classifiers was used to determine ion presence. A tie was decided in favor of a valid ion signal. For each signal identified as containing an ion signal, the FFT spectrum was then fit to the Lorentzian curve of Equation (6). Lastly, the signals were further filtered using the scheme described in Section 4.1, with somewhat arbitrary limits placed on the reasonable fit parameters: the Lorentzian height is decided somewhat arbitrarily based on the expected SNR, the Lorentzian width was restricted to $1.5\le c\le 5$ Hz, and the center frequency was restricted to $\pm 2$ Hz of the expected frequency. The effectiveness of these restrictions was evaluated on the collected experimental background data. The results for the 18,000 experimentally collected background signals showed that ≈10.5% were classified as containing ion data, but only ≈3.5% met the set filtering requirements and were incorrectly identified as ion signals.

6.2. Predicted Ion Characteristics

After the classification of the experimental signals discussed above, the resulting filtered signal areas were binned over an appropriate range of values with sufficient granularity and normalized. At this point the data were in the proper format for the neural network and can be seen in Figure 8 as the experimental input histogram. After performing the network regression, the predicted parameters were identified and are summarized in

Table 3. Neural network predictions on the experimental ⁸⁵Rb⁺ dataset. Summaries for each output parameter can be found in Section 5.2.

	Prediction
${\mu }_p$	0.26(6)
$A_{\mathrm{single}}$ (mV)	0.022(4)
$\sigma \left(A_{\mathrm{single}}\right)$ (mV)	0.0024(5)
Voffset (μV)	2.0(7)
${\varphi }_{0,\max }$ (rad)	4.3(1.4)

.

Following the procedure outlined in Section 5.3, a comparison of the input and reconstructed output is shown in Figure 8.

From the results presented in Table 3, one can extrapolate the percent composition for a given number of ions from the Poisson mean. However, as established in the prior section, there is a known non-zero false-positive classification rate of ≈3.5% which is not accounted for by the DNN. Recall that the Poisson probability relationship is given by

$$\begin{array}{cc}\hfill \mathrm{Pr}(X=k)= & \frac{{\mu }_p^k{e}^{-{\mu }_p}}{k!}\hfill \end{array}$$

(15)

$$\begin{array}{cc}\hfill \mathrm{Pr}(X=0)= & e^{-{\mu }_p}\hfill \end{array}$$

(16)

where X is the discrete random variable (number of ions), and ${\mu }_p$ is the Poisson mean. Using the fact that the ‘true’ percent of background signals is $(1+\mathrm{FPR})e^{-{\mu }_p}$, where $\mathrm{FPR}$ is the determined false-positive rate, the corrected Poisson mean can be shown to be

$$\begin{array}{cc}\hfill {\mu }_{p,\mathrm{corr}}= & {\mu }_p-ln(1+\mathrm{FPR})\hfill \end{array}$$

(17)

which is determined to be ${\mu }_{p,\mathrm{corr}}=0.23\left(6\right)$ for the measured FPR of $0.035$. Using this value, the expected percentages of ⁸⁵Rb⁺ signals produced by different numbers of ions in SIPT are calculated: $79\left(5\right)\%$ correspond to background, $18\left(4\right)\%$ are single-ion measurements, and $2.3\left(1.1\right)\%$ are measurements of at least two ions.

Additionally, the ion position in the trap and SNR can be extrapolated from the predicted $A_{\mathrm{single}}$ value using Equation (14), and were determined to be $\rho /{\rho }_0=0.9\left(3\right)$ and $\mathrm{SNR}=0.21\left(9\right)$, respectively, at a 95% confidence level. These values lie within the allowed range of values and are reasonable.

7. Conclusions

In this paper, we first outlined two methods for the simulation of FT-ICR signals and provided a qualitative comparison between a rigorous, but intensive, approach to a simplified form. The two methods were shown to behave near-identically, and we discussed the advantages of the simpler method for the needs of this work. Then, we provided novel applications of supervised machine learning techniques for FT-ICR signal and background signal classification. This approach was motivated by the need for an efficient filtering of signals, the majority of which are expected to be background for experiments concerning nuclei delivered at the lowest rates. Next, we described a multi-layer densely connected deep neural network capable of accurately predicting collective ion characteristics in SIPT when provided with a large set of similar same-isotope signal information. While not used to perform the mass measurement itself, this neural network shows promise as an invaluable diagnostic tool during an experiment.

We have also provided validation and analysis for the classification network using both numerically simulated ion signals and experimental background data. The low false-positive ion signal classification rates of ≈10.5%, and ≈3.5% following filtering, are critical for experiments with the lowest ion delivery rates. Additionally, for future experimental campaigns of ¹⁰⁰Sn⁺ the classifiers were shown to have accuracies of ≈65%, which are sufficient to successfully perform a mass measurement.

When evaluating the performance of the neural network, we qualitatively and visually compared the network input and reconstructed output for a unique sampling of numerically simulated datasets spanning a wide range of possible cases. Predictions of the most critical parameters ${\mu }_p$ and $A_{\mathrm{single}}$ in the simulated datasets showed agreement with the true value to $1\sigma$ in all but one case. Ultimately, we demonstrated that the network provided robust predictions that accurately described the simulated ion characteristics. When applied to an experimental dataset, the neural network suggests that SIPT is sensitive to single-ion signals, predicting that 18(4)% of the experimental signals resulted from single ions.

This initial work in machine learning applications for precision mass measurements using the FT-ICR technique leaves open several avenues for future developments and improvements. Further investigation into feature set extraction for signal classification, such as including additional data beyond the Fourier spectrum, may improve the classifier’s ability to distinguish noise and ion signals. Implementing ensemble classification may also improve classification precision when considering diverse feature sets. Further refining of the neural network structure and training characteristics is needed for improved extraction of ion information. Additionally, lifting potentially limiting assumptions requires a significantly expanded training dataset and further explorations for an optimal neural network structure.