Identifications of False Positives Amongst Sodium(I) Cations in Protein Three-Dimensional Structures—A Validation Approach Extendible to Any Alkali or Alkaline Earth Cation and to Any Monoatomic Anion

Abstract

Validation of the data deposited in the Protein Data Bank is of the upmost importance, since many other databases, data mining processes, and artificial intelligence tools are strictly grounded on them. The present paper is divided into two parts. The first part describes and analyzes validation methods that have been designed and used by the structural biology community. Everything began with the Ramachandran plot, with its allowed and disallowed types of backbone conformations, and evolved in different directions, with the inclusion of additional stereochemical features, distributions’ analyses of structural moieties, and scrutiny of structure factor amplitudes across the reciprocal lattice. The second part of the paper is focused on the largely unexplored problem of the high number of false positives amongst the sodium(I) cations observed in protein crystal structures. It is demonstrated that these false positives, which are atoms wrongly identified with sodium, can be identified by using electrostatic considerations and it is anticipated that this approach can be extended to other alkali and alkaline earth cations or to monoatomic anions. In the end, I think a global initiative, accessible to all volunteers and possibly overseen by the Protein Data Bank, should take the place of the numerous web servers and software applications by providing the community with a select few reliable and widely accepted tools.

1. Introduction

1.1. Overview of the Validation Tools

Validation of scientific observations is an essential aspect of every scientific procedure: it implies not only the estimation of the accuracy but also the estimation of the level of anomaly in the observation [1].

The Ramachandran plot marked the beginning of validation in structural biology [2,3]: by examining the values of phi and psi torsions, it is possible to determine if a protein’s 3D structure is normal or aberrant. Normal structures have phi and psi values within the permissible ranges, while abnormal structures have phi and psi values outside of these ranges. The validation technique was made accessible to structural biologists by the implementation of ProCheck, a computer application [4]: Google Scholar reports that the initial release of ProCheck, a publication that is almost 30 years old, has received over 27,100 citations.

Despite its user-friendly interface and clarity, ProCheck suffers from a significant drawback that is common among other similar software programs: it lacks the ability to differentiate between actual faults and functionally significant abnormalities, such as the accumulation of energy in the form of strain. In other words, ProCheck has the ability to detect abnormal backbone structures, even in situations where the abnormality is not necessarily a mistake.

Similar approaches were adopted in other computer programs, like AQUA and Procheck-NMR [5], ProSA [6,7], Molprobity [8,9,10,11], What_Check, and the use of RAMA-Z [12,13].

Additional structural features were included into the validation: for example, in MolProbity [8,9,10,11], the position of the hydrogen atom bound to the Calpha atom is computed and any potential collisions with other atoms are searched for; Afonine and his colleagues suggested utilizing the distribution of stereochemical hydrogen bond parameters to detect potential inaccuracies in low-resolution structures, where the reliability of Ramachandran plot analyses is reduced due to restraints imposed on backbone torsions [14].

Differently from the above methods, which primarily examine stereochemical features, SFCHECK [15] examines the structure factor amplitudes across the reciprocal lattice, assessing data completeness and the global and local agreement between the atomic model and experimental data.

In addition, there exist computational algorithms specifically developed to evaluate the quality of 3D models and choose the most credible option from a given set of options [16,17,18,19,20]. A novel non-parametric method has been tailored to detect outliers amongst the PDB entries [21] and a novel confidence score, derived from the real space correlation coefficients between residue coordinates and experimental electron density, was created to identify local anomalies within a single PDB entry [22]. Pereira and Lamzin proposed that Calpha atoms exhibiting atypical local stereochemistry can be distinguished by their positioning in a three-dimensional space defined by the first three orthogonal components computed using eigen analysis of the distance matrix of the Calpha(i − 1)—O(i − 1)—Calpha(i)—O(i)—Calpha(i + 1) atom locations [23].

While most validation strategies have focused on X-ray protein crystal structures, there have also been advancements in methods applicable to structures determined using other techniques. These include Procheck-NMR [5], which is specialized for solution NMR structures, the pioneering validation of results obtained from small angle scattering experiments [24], and EMRinger, a tool that assesses the fitting of an atomic model into the map of cryo-EM structures [25].

Concluding this concise and unavoidably incomplete review of 3D protein structure validation approaches, it is essential to remember the variety of methods employed in PDB to assess the accuracy of each entry [26,27]. Structures obtained by diverse experimental approaches, such as crystallography, solution NMR, and Cryo-EM, require the employment of distinct techniques. Crystalline structures, for example, are monitored using five main parameters: the free-R-factor, the frequency of outliers in the Ramachandran plot, the frequency of interatomic clashes in the backbone and sidechains, and the real space R-factor Z-score. Each parameter is compared to its calculated distribution in either the entire set of crystallographic structures or only in the subset of structures with similar resolution to the entry.

Three decades of efforts in developing protein structure validation tools have improved tremendously the quality of the data deposited in the PDB. For instance, I recall encountering a PDB entry about 30 years ago where the biological source was mistakenly labeled as “omo sapiens”, a simple typo that would likely have confused the search engines of that time. Nowadays, such an error would not happen.

1.2. Aberrant and Inaccurate Structures

However, there is still potential for improvements.

There exists a substantial body of work in the literature about mistakes in protein structures and many strategies that have been suggested for the purpose of discovering and rectifying them. Zbigniew Dauter and colleagues wrote in 2014 that “Common errors can be traced to negligence and a lack of rigorous verification of the models against electron density, creation of non-parsimonious models, generation of improbable numbers, application of incorrect symmetry, illogical presentation of the results, or violation of the rules of chemistry and physics” [28].

After one year, it was noted that the majority of protein complexes with the anticancer medicines cisplatin and carboplatin exhibited substantial issues, either related to crystallography or chemistry [29]. Analogously, wrong zinc(II) sites were discovered in several protein structures [30]. It was discovered that, in a few structures of metallo-beta-lactamase, the interpretation of the bound ligands (e.g., inhibitors, substrate/product analogs) is doubtful or even incorrect [31].

For obvious reasons, the quality of the structures of proteins related to COVID-19 has been carefully checked [32,33,34]. Also, the structures of L-asparaginases, enzymes that include approved drugs for the treatment of certain types of leukemia, were verified [35]. Analogously, the quality of large-scale crystallographic fragment screening projects, the purpose of which is the identifications of very low-occupancy small molecule ligands in macromolecular complexes, has been questioned [36]. A systematic analysis of the quality of structures from structural genomics was published too [37].

Other issues are important and largely unexplored. For example, the identification of abnormal B-factors, whether they are isotropic, anisotropic, or produced by TLS refinements, by examining B-factor expected values and distributions, by considering various features like resolution, crystal packing, solvent accessibility, etc. Another issue is related to metal cations, such as sodium(I) or potassium(I), and monoatomic anions, such as chloride or bromide, which are occasionally misidentified.

This communication specifically addresses the latter problem. There are mistakes, as all of us know. For example, it has been reported that several metal cations are completely isolated in the crystal lattice, with no atoms bound to them: they are totally naked, something that is simply absurd [38].

Here, I will not provide a comprehensive method for evaluating the chemical nature of metal cations and monoatomic anions in protein structures. From my perspective, while this endeavor is intriguing and has the potential to be valuable to the scientific community, it is not suitable to be presented as one of the numerous (maybe excessive) computer applications and servers that exist on the internet. Here, I present a proof of concept about the ability to accurately and reliably identify electron density peaks that may have been mistakenly identified as metal cations or monoatomic anions. I would like to propose that the PDB initiates a research program, open to the scientific community, aimed at preventing cation and anion assignment errors, if not all of them. In fact, as reported by Rupp and colleagues, ensuring the non-publications of structures containing mistakes should be the outcome of a collaborative effort [39,40].

Such an initiative for validating metal cations and monoatomic anions can be grounded, of course, on the literature in this field, relatively abundant for metal cations and less for anions, where there are numerous studies and quite a few databases [41,42,43,44,45,46,47,48,49,50,51,52,53].

Significantly, numerous attempts have been made to validate metal biosites in metalloproteins [43,54], where the cations can perform diverse functions such as adjusting thermodynamic stability, enabling oxygen transport and storage, and catalyzing chemical reactions. Conversely, the validation of metal cations or monoatomic anions that occur by chance in the structure due to their presence in the crystallization mixture remains mainly unexplored. From a purely biological standpoint, these cations and anions are of lesser significance. Nevertheless, it is advisable to pretend that they are accurately designated.

This study investigates the columbic potential energy of the chemical environment around sodium(I) cations. The analysis is conducted on ten subsets of protein crystal structures that have been carefully selected from the PDB database. The potential energy distributions enable the identification of outliers that may not be a sodium(I) cation but rather something distinct. It is expected that the same method can be readily applied to other alkali or alkaline earth cations and monoatomic anions.

This approach has the advantage of being useful non only in crystallography, where, in many cases, the chemical nature of the atoms can be inferred from anomalous scattering, but also in other structural biology methods (Cryo-EM, NMR), where, in general, there is no way to make qualitative analyses about the chemical nature of the atoms.

Preventing these errors is not a trivial focus on minor but still significant details, as anything that is stored in the Protein Data Bank, and subsequently in other databases that use its data, can contaminate the results of statistical analysis, particularly in the era of artificial intelligence. The issue of bias in data mining the Protein Data Bank (PDB) caused by varying degrees of inaccuracies in protein structures has been extensively documented [55].

Misidentifying sodium(I) cations within three-dimensional protein structures can have significant consequences, introducing noise in the analysis of protein hydration layers and potentially altering the local protein surface. These errors can negatively affect docking studies and virtual screening outcomes, making it crucial to minimize them as much as possible.

An effective analogy may be drawn from the culinary world: a meticulously refined 3D structure can be likened to a Michelin-starred restaurant, whereas a sloppy structure resembles a university cafeteria where the dishes are merely palatable but lack distinction. Alternatively, one might highlight the difference between a mass-produced prêt-à-porter dress and a custom-made haute couture dress.

It is worth remembering how a sense of accomplishment can rescue people from the most heinous situations, as Primo Levi showed in “Se questo è un uomo [If This is a Man]” [56,57]. Levi tells the story of Lorenzo, an Italian mason imprisoned in a Nazi concentration camp who, despite the awful circumstances, continues to build a wall with commitment “solo per il piacere di fare un lavoro ben fatto [just for the satisfaction of doing a job well]” as a symbolic act of defiance against the camp’s atrocities. The same should happen in science.

2. Results

All protein crystal structures containing sodium(I) cations were downloaded from the Protein Data Bank and ten subsets were randomly assembled, each containing 100 structures (see Table S1 in the Supplementary Material), according to the procedure described by Carugo [58]. Thus, one may concurrently perform 10 analyses and estimates, then average the results. This enables the calculation of standard errors for each average, so assessing the reliability of the predictions.

The n non-hydrogen atoms within 3.5 Å from the sodium(I) cation were retained, their electrical charges q_i were taken from the CHARMM force field (see Table S2 in the Supplementary Material; [59]), and the coulombic potential energy V of the sodium(I) cation was computed as

$$V=\sum _{i=1}^n{\displaystyle \frac{q_i}{d_i}}$$

where d_i is the distance of the i-th atoms from the sodium(I) cation. Crystal packing was explicitly considered, as described by Djinovic-Carugo and Carugo [38], to include atoms belonging to adjacent asymmetric units.

The threshold of 3.5 Å is slightly larger than the sum of the sodium(I) ionic radius and the oxygen van der Waals radius (2.5–2.9 Å). This is necessary for taking into account the intrinsic inaccuracies in protein crystal structures, especially at the protein surface where misidentifications are more probable, and to guarantee adequate coverage of the volume surrounding the cation.

Sodium(I) cations where it was impossible to assign the electric charge to one of their neighbors were discarded.

A sodium(I) cation surrounded by negatively charged atoms is expected to have V < 0. On the contrary, a sodium(I) cation with V close to 0 or even >0 is expected to have been erroneously identified and it is probably not a sodium(I) cation and, perhaps, it is not a cation at all.

A list of all sodium(I) cations and of the atoms close to them is available on request.

Figure 1a illustrates the distributions of the Coulombic potential energy Vs across the ten data subsets. The mean distribution, calculated by averaging the ten subsets, is shown in Figure 1b. The ten distributions are, with minor deviations, similar, with a maximum around −1.1. The same phenomenon is seen in the averaged distribution. Very few values of V are found below −2 (4.37%) or above 0 (0.06%).

These data may be used to assess a novel sodium(I) cation. If its V is higher than 0, it may be classified as an outlier with a 99.94% probability, since just 0.06% of the sodium(I) cations in the PDB have V values above 0.

To enhance the efficacy of this method, it is advisable to evaluate the new sodium(I) cation inside each of the ten subsets. The probability to be an outlier must be computed in each subset and, subsequently, the mean probability may be calculated.

Figure 2 illustrates three examples of atoms deposited as sodium(I) cations, but they may likely be totally different things. The randomly chosen instances are not meant to denigrate the authors but only to demonstrate how easily such mistakes can arise, especially when the emphasis on structure is mostly biological and little attention is paid to apparently trivial aspects.

In the first case (Figure 2a), a sphere with a radius of 3.5 Å, centered on the cation, encompasses a carboxylic side-chain oxygen atom, a backbone oxygen atom, a backbone carbon atom, and two backbone nitrogen atoms. The Coulombic potential energy (−0.534) is rather elevated, as deduced from a comparison with the data shown in Figure 1. Quantitatively, just 8.74% of the sodium(I) cations from the ten subsets shown in Figure 1 possess Coulombic potential energy values above −0.534. This indicates that the cation shown in Figure 2a is likely not a sodium(I) cation and, presumably, is not a cation at all.

In the two further cases (Figure 2b,c), even the termini of lysine side-chains being within 3.5 Å of the sodium(I) cations and the Coulombic potential energy values are greater than in the first example (−0.414 and −0.444). Furthermore, only 6.54% of the sodium(I) cations from the ten subsets shown in Figure 1 exhibit Coulombic potential energy values exceeding these values. This indicates that an incorrect assignment has occurred, implying that most likely these atoms are not sodium(I) cations.

Conversely, Figure 3 presents two instances in which the sodium(I) cations have been almost definitely accurately identified. In both cases, they are at a crystal packing contacts and they are coordinated by carboxylate side-chains and by several water molecules. Their Coulombic potential energy values (−2.05 and −2.12) are low and the large majority of the sodium(I) cations from the ten subsets shown in Figure 1 exhibit Coulombic potential energy values exceeding these values.

These examples unequivocally demonstrate that anomalous cations may be identified in PDB entries by a simple method grounded on electrostatic interactions.

3. Discussion

Errors in identifying sodium(I) cations within three-dimensional protein structures can lead to substantial repercussions, as they introduce noise in the analysis of hydration layers surrounding proteins and may modify the local structure of the protein surface, adversely affecting docking and virtual screening results. It is, consequently, important to minimize these errors as much as possible.

Clearly, the strategy outlined above can be improved, for instance, by include hydrogen atoms, which are currently disregarded, despite their locations often being indeterminate. Nonetheless, this raises a debate of the extent to which nonexperimental knowledge might be used in the validation of experimental results.

Also, unusual B-factor values may indicate misassignments of sodium(I) cations, which could be more accurately interpreted as heavier or lighter elements or groups, or may suggest that the occupancies are <1 [63,64].

Moreover, the percentages calculated in Figure 2 are likely overestimated. In fact, several sodium(I) cations included in the ten subsets of Figure 1 are likely not authentic sodium(I) cations and may be false positives and, consequently, the proportion of authentic sodium(I) cations with Coulombic potential energy above that shown in Figure 2 is likely less than 8.74% or 6.54%. This reinforces the warning that the examples shown in Figure 2 are outliers and, hence, do not represent sodium(I) cations. Nonetheless, any a priori assumption used to eliminate false positives from the ten subgroups shown in Figure 1 has inherent risks, since it lacks direct experimental evidence.

It is prudent to eliminate any isolated sodium(I) cations. These cations lack nearby atoms and have been described as “naked metal cations swimming in protein crystals”. These are probable mistakes in the interpretation of the electron density maps, in the absence of additional experimental evidence.

These considerations imply an additional remark.

The performance of prediction algorithms is often assessed by whole or partial cross-validation. This necessitates the creation of two data sets, one designated as positive and the other as negative, followed by a determination of the amounts of true positives, true negatives, false positives, and false negatives. Thus, with these four integer values, one may evaluate the accuracy of the predictive algorithms together with their sensitivity, specificity, and Matthews correlation coefficient.

The first phase of this approach is obstructed here, since it is infeasible to create two distinct datasets: one including metal sites that are unequivocally sodium(I) sites and the other consisting of metal sites that are definitively not sodium(I) sites. Therefore, I am unable to provide strong and accurate assessments of the success of the prediction approach outlined in this report. This strategy may assist in preventing errors: sodium(I) candidates with substantial Coulombic potential energy should be regarded with caution, particularly when the proportion of occurrences with greater energy is minimal.

4. Conclusions

Validation of results is fundamental to science, and in structural biology, it involves identifying errors and inaccuracies in macromolecular structures. For over three decades, significant efforts have been made in this area, addressing various and different aspects such as the analysis of backbone phi and psi torsions and the distribution of structure factors.

It is impossible to eliminate all potential errors in structural biology as in any other human endeavor. However, it is possible to enhance our technology to reduce mistakes to a minimum. Interestingly, minor and major mistakes occur in crystallographic analyses of small molecules, where the ratio between experimental information and number of parameters to be determined and refined in much higher than in protein crystallography. Several examples can be found in the literature, from the pioneering works of Marsh, who highlighted erroneous space group determinations (see for example [65,66,67]; for a long time, the expression “to be marshed” indicated someone who published a wrong structure), to more recent publications (see for the numerous examples reported in [68,69]).

Someone with a chemical background will readily observe in Figure 2 that the atoms listed in the PDB as sodium(I) cations are not actually sodium(I) cations, nor can they be classified as any other form of cation. However, inexpert structural biologists or scientists lacking proper education in crystallography and structural chemistry might not realize this mistake, especially if under stress to publish quickly to secure a tenured position, spurred on by greedy faculty deans.

In addition to using validation methods, the responsibilities of the reviewers must be reinforced. Numerous protein crystal structures, particularly those deemed “hot” within the molecular biology community, are published in scientific journals that emphasize biological novelty above structural findings. As a result, it is possible that none of the reviewers has expertise in crystallography, NMR, or CryoEM. Consequently, erroneous structures may be disseminated rather being refined under the reviewers’ supervision. It is preferable that at least one reviewer have extensive experience in crystallography or relevant areas of structural biology as needed.

Many computer programs have been published during the last three decades in the field of protein structure validation. Although the Protein Data Bank has implemented a rigorous validation strategy using a limited set of tools, several other methods are available, which can result in some uncertainty and disorientation. In my opinion, the creation of a more unified and standardized approach is advisable.

This initiative, potentially coordinated by the Protein Data Bank, should welcome contributions from all scientists and remain flexible, allowing for the incorporation of new validation methods and assessments of new structural features, such as the false sodium cations discussed in this communication. Additionally, it should be adaptable to structures determined not only by X-ray crystallography, solution NMR, and cryo-EM, but also by future technologies.