Next Article in Journal
Current Perspectives on the Use of off the Shelf CAR-T/NK Cells for the Treatment of Cancer
Previous Article in Journal
Palmitoylethanolamide Reduces Colon Cancer Cell Proliferation and Migration, Influences Tumor Cell Cycle and Exerts In Vivo Chemopreventive Effects
Previous Article in Special Issue
Liquid Biopsies in Solid Cancers: Implementation in a Nordic Healthcare System
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Using the R Package Spatstat to Assess Inhibitory Effects of Microregional Hypoxia on the Infiltration of Cancers of the Head and Neck Region by Cytotoxic T Lymphocytes

1
Department of Radiation Oncology, University Medical Center of the Johannes Gutenberg University, 55131 Mainz, Germany
2
Department of Mathematical Sciences, Aalborg University, Skjernvej 4A, 9220 Aalborg East, Denmark
3
Edinburgh Pathology, Institute of Genetics and Molecular Medicine, University of Edinburgh, Edinburgh EH4 2XR, UK
4
Institute of Pathology and Tissue Biobank, University Medical Center of the Johannes Gutenberg University, 55131 Mainz, Germany
*
Author to whom correspondence should be addressed.
Cancers 2021, 13(8), 1924; https://doi.org/10.3390/cancers13081924
Submission received: 4 March 2021 / Revised: 27 March 2021 / Accepted: 12 April 2021 / Published: 16 April 2021

Abstract

:

Simple Summary

Progress in the field of in situ proteomics allows for the simultaneous detection of multiple biomarkers within one cancer tissue specimen. As a result, biological hypotheses previously only assessable ex vivo can now be studied in human cancer tissue. However, methods for objective analysis have so far been lacking behind. In this study, we established a free, objective, and entirely open-source-based method for the analysis of multiplexed immunofluorescence specimens. This will gain further importance with the availability of more advanced multiplexing methods in the future.

Abstract

(1) Background: The immune system has physiological antitumor activity, which is partially mediated by cytotoxic T lymphocytes (CTL). Tumor hypoxia, which is highly prevalent in cancers of the head and neck region, has been hypothesized to inhibit the infiltration of tumors by CTL. In situ data validating this concept have so far been based solely upon the visual assessment of the distribution of CTL. Here, we have established a set of spatial statistical tools to address this problem mathematically and tested their performance. (2) Patients and Methods: We have analyzed regions of interest (ROI) of 22 specimens of cancers of the head and neck region after 4-plex immunofluorescence staining and whole-slide scanning. Single cell-based segmentation was carried out in QuPath. Specimens were analyzed with the endpoints clustering and interactions between CTL, normoxic, and hypoxic tumor areas, both visually and using spatial statistical tools implemented in the R package Spatstat. (3) Results: Visual assessment suggested clustering of CTL in all instances. The visual analysis also suggested an inhibitory effect between hypoxic tumor areas and CTL in a minority of the whole-slide scans (9 of 22, 41%). Conversely, the objective mathematical analysis in Spatstat demonstrated statistically significant inhibitory interactions between hypoxia and CTL accumulation in a substantially higher number of specimens (16 of 22, 73%). It showed a similar trend in all but one of the remaining samples. (4) Conclusion: Our findings provide non-obvious but statistically rigorous evidence of inhibition of CTL infiltration into hypoxic tumor subregions of cancers of the head and neck. Importantly, these shielded sites may be the origin of tumor recurrences. We provide the methodology for the transfer of our statistical approach to similar questions. We discuss why versions of the Kcross and pcf.cross functions may be the methods of choice among the repertoire of statistical tests in Spatstat for this type of analysis.

Graphical Abstract

1. Introduction

Squamous cell carcinomas of the head and neck often contain extensive tissue areas with substantially reduced oxygen partial pressures compared to the corresponding normal tissues [1]. Severe tumor hypoxia has long been recognized as a predictive factor for an inferior response to primary radiotherapy in this tumor entity [2]. Not only is the effect of radiotherapy under fully oxygenated conditions amplified by up to a factor of three (oxygen enhancement effect) compared to hypoxic/anoxic conditions, there are numerous other implications relevant to therapeutic resistance [3,4].
In recent years, the success of immune-checkpoint inhibitors has brought the immune system’s role in tumor therapy into the focus of attention. E.g., it has been shown that the immune system also partially mediates the effects of radiotherapy of cancers of the head and neck [5]. Various studies in different tumor entities have demonstrated that the density of the infiltration by CD8-positive cytotoxic T lymphocytes (CTL), and especially the pattern of this infiltration, represent vital prognostic factors [6,7].
Hypoxic tissue areas in experimental tumors have been found to contain low CTL, suggesting that hypoxic tissue regions might be partially protected from the immune system’s access [8]. Conversely, hyperoxia reactivated the immune system in the tumor microenvironment and led to the regression of metastases in experimental animals [9]. However, one difficulty in testing such a hypothesis is that the assessment of infiltration patterns is highly subjective and, therefore, associated with questionable reproducibility. Objective classifications and quantification of the tumor immune phenotype would be desirable, especially if future therapy decisions are based on such assessments.
The spatial statistics methods include tools already established in other areas of science (e.g., ecology, geosciences). They seem promising candidates to enable an objective examination of cell distribution and interaction patterns in human tissue as well. This has become possible only recently due to the availability of fluorescence slide scanners and analysis software, allowing single-cell segmentation of cells and subsequent classification into different tumor compartments (e.g., tumor cells, stromal cells) based on machine learning algorithms. In the present work, we have applied spatial statistical methods implemented in the R package Spatstat (E.R. & C.A.N.B., http://spatstat.org/) (accessed on 13 April 2021) [10] to squamous cell carcinomas of the head and neck region and compared the results with a traditional visual analysis of CTL infiltration patterns. Data acquisition in 22 specimens was performed in the open-source software QuPath (P.B., https://qupath.github.io/) (accessed on 13 April 2021) [11].

2. Materials and Methods

2.1. Selection of Specimens for the Study

A total of 103 whole-slide head and neck tumors (oral cavity, nasopharynx, oropharynx, hypopharynx, larynx) from the Biobank collection of the University Medical Center Mainz and one microarray with a total of 70 spots (US Biomax, Derwood, MD, USA, Cat.-No. HN-801a) were available for the study. Regarding the latter, some spots showed an insufficient degree of morphological preservation and could not be evaluated. Finally, we included a total of 22 specimens in the analysis, which contained both extensive hypoxic micro-regions and sufficient infiltration by CTL.

2.2. Immunofluorescence Staining

Slides were stained for the antigens carbonic anhydrase (CA) IX, CD8, and CD73. Only the former two antigens are relevant for this study. Results from the analysis of CD73 will be communicated soon (manuscript in preparation). The 4-plex staining protocol has been described in detail before [12]. In brief, we heated the specimens in 10/1 mm Tris/EDTA pH 9.0 buffer in a microwave oven (Bosch HMT75M451, Robert Bosch GmbH, Munich, Germany) for 15 min to retrieve antigenic binding sites. Next, we incubated tissue specimens with the primary antibodies listed in Supplementary Table S1. We used the biotin-free, micro polymer-based SuperPicture™ reagents (Thermo Fisher Scientific, Waltham, MA, USA) directed against the species of origin of the primary antibodies to label the latter with horseradish peroxidase. We achieved the visualization of antigens by incubating specimens with appropriate fluorochrome-tyramide conjugates in appropriate buffers after their activation with H2O2 (see Supplementary Table S1). We repeated microwave treatment between successive rounds of fluorescent staining to quench peroxidase activity from preceding antigen-labeling steps. In a final step, we incubated the slides with 4′,6-diamidino-2-phenylindole (DAPI) at a concentration of 300 nM. Finally, slides were covered with a coverslip using DAKO Fluorescence mounting medium (Dako Deutschland GmbH, Hamburg, Germany) and dried overnight at 4 °C.

2.3. Acquisition of Digital Whole-Slide Images

Slides were digitized as czi files using the Zeiss AxioScan Z.1 and appropriate single-band filter sets for the fluorescence channels, DAPI, fluorescein isothiocyanate (FITC), Cy3, and Cy5 (Morphisto GmbH, Frankfurt, Germany). For visualization and single cell-based segmentation of the digital tissue specimens, we used the open-source software QuPath (https://qupath.github.io/) (accessed on 13 April 2021) in conjunction with a built-in Bioformats extension to decode the *.czi file format.

2.4. Visual Assessment

Visual assessment of the specimens and the determination of the threshold levels for the hypoxia marker (i.e., hypoxic vs. non-hypoxic cells) occurred after adjustment of the gray levels to optimize visibility of individual color channels. This introduces a minor subjective component that cannot be eliminated on this technological platform. An example of a flat image without adjustments and the same image after adjustments to improve visibility is given in Supplementary Figure S1. Specimens were assessed visually for clustering of CTL and subsequently categorized based upon our (J.K., A.M.) subjective estimate of the nature of the interaction between CTL and hypoxic tumor tissue into negative interaction (inhibition), no interaction, and positive interaction (clustering).

2.5. Data Preparation for Spatial Statistics in Spatstat

As a first step, we chose representative tissue regions of interest (ROIs) containing approximately 20,000 cells. For the whole-slide specimens, a rectangular region with a prominent expression of CA IX and CD8 was chosen. The environment of all of the ROIs is shown in Supplementary Figure S2. With regard to the tissue microarray spots (n = 6), the analysis comprised the entire spot. Within these ROIs, every cell was segmented using a watershed-based algorithm implemented in QuPath. Subsequently, we programmed QuPath to classify cells into tumor and stroma using QuPath’s machine learning tools and a random-forest method. After that, we set thresholds for the antigen intensities interactively. This was necessary to account for inter-slide and inter-patient variability in fluorescence intensity resulting from differences in tissue handling (e.g., differences in fixation time). In total, we generated four classes of events corresponding to the following subpopulations of cells: 1 = CTL, 2 = normoxic tumor, 3 = hypoxic tumor, 4 = stroma. Finally, we exported the single-cell data from QuPath into a *.txt file, imported this file in R (using read.delim), and analyzed data using the R package Spatstat (http://spatstat.org/) (accessed on 13 April 2021). In Spatstat, every cell of the original image was represented as a point event in a planar (i.e., two-dimensional) point pattern belonging to an ROI derived from the original cell X-Y coordinates in the QuPath dataset. Subsequently, every event was marked using the class of the cell it represents. Figure 1 gives an overview of the entire procedure. To achieve better visibility for Figure 1, we chose a slightly smaller ROI with a lower number of cells (n = 12,635). Point patterns, which are subdivided by a categorical class variable (a factor in Spatstat terms), are referred to as multitype point patterns. Spatstat provides several tools for the analysis of the interaction between populations in multitype point patterns. Different statistical functions can be used for interaction analyses in multitype point patterns of this kind. The significance level was set to alpha = 5% for all computations. Our analysis showed that versions of the K and pcf functions might be the most suitable to the pathophysiological question of our study (see Results).

3. Results

Experimental studies have shown that CTL, in principle, can freely infiltrate tumor tissue, i.e., tumor and stroma areas, including the interstitial space between cells in tumor cell aggregations [13]. Figure 1A depicts an example of CTL distribution in a specimen from this study in which CTL are both observed within the tumor stroma and inside the tumor cell islets. Hence, we first asked whether CTL distribution shows systematic deviation from complete spatial randomness (CSR) in the form of clustering. Visual inspection strongly suggested that this was indeed the case in all specimens, and CTL seemed to cluster in the stroma (Figure 2).
A number of functions in Spatstat can be employed to assess the question of deviation from CSR. First, the quadrat test subdivides the ROI into a set of rectangles and tests for deviations from a homogenous intensity between them [14]. As expected, the quadrat test, in our example, indicated inhomogeneity (chi-square test, p < 0.0001, Figure 2A). This is important when choosing the right functions in the subsequent parts of the analysis. To adjust for the inhomogeneity shown by the quadrat test, inhomogeneous versions of the functions have to be used. These try to adjust for the inherent inhomogeneity in the pattern, which might suggest a false clustering effect otherwise. Next, we used the inhomogeneous G-function (Ginhom), which calculates the nearest neighbor probabilities, i.e., the chances of a random point of a point pattern to find its nearest neighbor within a given distance [15]. With an increasing radius around a given point in the pattern, the G-function value can only ever increase and has a theoretical maximum of 1 (i.e., 100% likelihood). When compared with the G-function of a random distribution (i.e., a Poisson point process), the G-function showed the tendency of CTL to have a higher than random chance to encounter the next CTL within a distance of 8–23 µm (Figure 2B). These analyses were repeated for the remaining 21 specimens in the study. Hence, the clustering of CTL was unanimously supported by the inhomogeneous G-function.
More advanced statistical tests enable the assessment of a deviation from spatial randomness on a continuum of radii in the two-dimensional space. The functions in Spatstat that we have employed to achieve this are Ripley’s K-function (Kest) [16] and the pair correlation function (pcf) [17,18]. Since the population of CTL was found to be distributed inhomogeneously throughout the tumors (see above), we used the inhomogeneity-corrected versions of the K-function and the pcf (Kinhom, pcfinhom) to adjust for spatial inhomogeneity [19]. As expected, we could confirm the clustering of immune cells using these functions in the previous example (Figure 2C,D). There are several reasons for this, which will be discussed (See Discussion).
To demonstrate the difference between various distribution patterns and how they might influence the K-function or the pcf, C.A.N.B. and J.K. have started to work on two Shiny-web applications that are still under further development. One showcases different patterns and their corresponding spatial statistical functions (Figure 3A–C). When comparing these different patterns to the distribution pattern of CTL, the clustering nature of CTL seemed obvious. The second app was created to provide a low-effort introduction toward experimenting with spatial statistics. Researchers can upload their own single-cell datasets and generate results. The beta versions of the apps can be accessed here (http://apps.math.aau.dk/spatstat/) (accessed on 13 April 2021).
We subsequently applied all of the previously mentioned statistical tests to the entire dataset of the study and found evidence for clustering of CTL in all cases using the inhomogeneous versions of the G-function (data not shown) and the pcf (Supplementary Figure S3). To test for significance, we used the lohboot function, which calculates confidence intervals for the functions using a bootstrap method. The inhomogeneous K-function only implied clustering in 18 out of 22 cases (82%) in the study (Supplementary Figure S4). This finding is most likely caused by the cumulative nature of the K-function, resulting in a tendency to underestimate the significance of a deviation from spatial randomness (see Discussion section). When all functions resulting from a single ROI are considered before making a statement about the distribution of a point pattern, there is a high implication of clustering in all specimens.
We next turned to the second and more challenging question of possible systematic avoidance of hypoxic areas by CTL. Visual inspection suggested this to be the case in only 9 of 22 patients. To assess this question using Spatstat tools, we first considered the multitype equivalent of the G-function, the Gcross function. However, we refrained from using the Gcross function for the question at hand since it is not suitable for this task due to its lack of standardization (see Discussion section). Hence, we next employed the multitype equivalents of Kest and pcf, the Kcross and pcf.cross functions using two different pairs of cell types, i.e., CTL versus normoxic (1 vs. 2), and CTL versus hypoxic tumor cells (1 vs. 3) [20,21]. Due to the inhomogeneous nature of the CTL distribution and hypoxia in the tissue, we once more employed the inhomogeneity-corrected variations. In addition, we again used the lohboot function to test the significance of possible deviations of the interaction between these pairs of cell types from complete spatial randomness. For 16 out of 22 cases, both tests showed a significant divergence of the tests’ confidence intervals for the two populations, directed in a way that implied an inhibitory effect on CTL by hypoxia. Although not statistically significant, we found trends toward a repellent effect of hypoxia on CTL in all but one of the remaining specimens (examples are shown in Figure 4).

4. Discussion

In the present study, we have demonstrated that CTL in squamous cell carcinomas of the head and neck region exhibit clustering and avoid hypoxic areas upon infiltration into the interstitial space between cells inside tumor cell aggregates. CA IX was chosen as an endogenous marker of hypoxic tissue areas because it has been shown to co-localize with the gold standard exogenous marker pimonidazole [22] and because the antigen has a prolonged half-life after hypoxia exposure, thereby integrating biologically relevant episodes of hypoxia over longer periods of time [23]. Since this finding has potentially broad pathophysiological significance, e.g., concerning the interaction between radiotherapy and immunotherapy, a factual methodological basis for such a statement is essential. Although the phenomenon of cluster formation seems easy to judge visually, there are borderline cases in which doubts about such behavior can arise. The expenditure for statistical test procedures in Spatstat is, as shown here, low and provides certainty in less obvious cases.
The assessment of the distribution of CTL within the tumor cell aggregates is inherently more subjective. In most specimens, it is not obvious that CTL may avoid hypoxic areas. In fact, in more than half of the cases, our visual analysis suggested that CTL did not show such behavior. It is well documented that the visual processing of patterns in the human brain is distorted by the natural attempt to generate meaning [24,25]. It may, therefore, be flawed. Specifically, there is an incentive to detect patterns consistent with a scientific study’s underlying hypothesis, even though they may not exist biologically. This is related to the phenomenon of cognitive dissonance, which describes the tendency to filter information [26]. In this setting, objective statistical analyses demonstrate their strengths and offer reproducible, investigator-independent results.
At this point, it may be essential to consider the differences between inhomogeneity and clustering. While both result in similar patterns, the reason for these patterns is vastly different. While inhomogeneity results from external factors, e.g., more favorable conditions, clustering results from internal factors or actual interaction between individual entities such as cell-to-cell-signaling. It is important to note that there is no need for direct contact between two cells to interact, as this interaction can occur via soluble factors such as chemokines. Furthermore, the scale of interaction is essential as well, e.g., when viewed at the scale of our galaxy, humans seem to show strong clustering. However, when the scale is changed to a city, then the subjective evaluation of the spatial distribution of humans is reversed. They are now more likely to be perceived as repelling each other.
In short, clustering on a large scale is more synonymous with inhomogeneity, while clustering on a small scale can be interpreted as actual clustering. Although challenging, defining the correct scale for interaction is, therefore, of utmost importance. For this study, we arbitrarily limited the size of the scale to 100 µm.
While the functions used may seem straightforward, there are a few things to be considered during the interpretation of results. The quadrat test might be a quick and easy tool to obtain a first impression of whether a point pattern is distributed randomly or not. Still, it can only detect deviations from the null hypothesis of a Poisson distribution. The cause and direction of the deviation, however, cannot be determined. Tools for additional analysis of the quadrat test may be available in the future. When analyzing the distribution of a single population, the G-function is a useful tool to detect deviations from CSR.
In the second part of our analysis, however, we compare the effects of two different populations (normoxic and hypoxic tumor cells) on a third population (CTL). To do so, we compare the results of the Cross-functions between each of the first two populations and CTL. However, the usage of the Gcross function for this question is not straightforward. It has a few pitfalls, the most important one being that it is not standardized, and therefore the results are dependent on the intensity of a point pattern. Graphs of two Gcross-functions resulting from different combinations of populations should, therefore, not be compared. Although the K-function and the pcf also have limitations, the ability to be used for inter-group comparisons is a strength of both; since they can be adjusted for the intensity of a point pattern, it is possible to compare the degree of deviation from spatial independence of the two cell populations investigated. Problems of the K-function lie within its cumulative nature. While this makes the K-function more robust for statistical inference, such as the lohboot function, it also causes a slower reaction to actual deviations from CSR present in the specimen. This can suggest that effects take place over a more extended scale of distances than is biologically the case and make them seem not as significant as they might be, resulting in a loss of statistical power on the local level. On the other hand, this cumulative nature and the more global approach make the K-function easier to estimate and, as a result, more robust for statistical inference, such as the lohboot function on a global scale.
The pcf, conversely, is not cumulative and has more local information. Compared to the K-function, it is superior when analyzing the deviations from CSR and on what scale they do take place. However, it is more difficult to estimate and may suffer from high variance, resulting in reduced statistical power globally. This can be seen in our study as well.
When evaluating the statistical results, we noticed that the inhomogeneous K-function did not significantly deviate from CSR in some of the specimens. The reasons for this are manifold. On the one hand, in specimen with a sparse immune cell infiltrate, the K-function show significantly wider confidence intervals due to smaller case numbers. Therefore, the deviation from CSR must be substantially larger to be considered significant. On the other hand, one must consider that the K-function indicates a significant departure from CSR over an entire distance.
In contrast, the pcf-function does so for a specific distance. Hence, deviations from CSR, which are recognized as significant by the pcf, do not necessarily lead to a deviation of the K-function due to an averaging effect of the entire area in which the cells are counted. Conversely, a weak effect over a longer distance may not cause the pcf to indicate a significant deviation from CSR, while the K-function may do this. Additionally, due to the cumulative nature, effects within a smaller distance may cancel out effects taking place on a bigger scale and result in the K-function not deviating from CSR.
Furthermore, attempts are made to reduce the number of false-positive results by correcting the K-function for, e.g., boundary effects and inhomogeneity. A lack of significance of the K-function does not necessarily contradict the statement that clustering occurs within a population. Looking at the curves of the corresponding pcf, however, it appears that there is only comparatively mild clustering in these particular specimens. The reasons for this could be an unfavorable environment for the CTL or other factors not tested here. Future comparisons of data from these statistical tests with survival data may identify biological differences between different types of clustering behavior of CTL [27].
While overcoming most of the subjective parts of this kind of analysis, the statistical methods presented here do not yet enable a fully automated, unbiased evaluation. Such a methodology would certainly be very interesting as it would open up the possibility of cell distribution assessment in large numbers of specimens without user interaction.
Interestingly, despite the K-function’s lack of sensitivity described above, results obtained with the cross variant of the function were surprisingly unanimous in the second part of our analysis. According to the hypothesis of hypoxia-induced immunosuppression, we expected the Kcross and pcf.cross functions to show more substantial deviations from CSR for the comparison between hypoxic tumor cells vs. CTL with normoxic tumor cells vs. CTL. Indeed, this pattern could be demonstrated for all specimens in the study (see Figure 4 and Supplementary Figure S5).
Fluorescence-based multichannel staining of formalin-fixed and paraffin-embedded human tumor tissue in conjunction with whole-slide scanning, digitization, and single cell-based segmentation now offers the opportunity for mathematical analyses of human cancer tissue specimens. With the emergence of even more advanced biomarker analysis methods with the possibility of higher multiplexing, the necessity of using such methods will become increasingly apparent. Simultaneously, the data’s validity and reproducibility will improve, and the opportunity will arise to ask more complex questions than is possible with the limited number of antigens today. Four advanced systems are nanostring digital spatial profiling (nanostring DSP, [28]), imaging mass cytometry (IMC, Fluidigm [29]), multiplexed ion beam imaging (MIBI, IonPath [30]), and Cell DIVE (Cytiva, formerly General Electric [31,32]). The methodological approaches differ considerably: while Cytiva’s system uses conventional fluorescence, the two mass spectrometry techniques (IMC, MIBI) work with lanthanide-coupled antibodies, while nanostring technology uses short nucleic acid sequences that can be coupled to antibodies or RNA molecules like a barcode. Simultaneous detection of more than 30 proteins has been published for all methods, but the theoretical maximum numbers are higher. So far, the highest published number of antigens detected simultaneously was achieved on the Cell DIVE System in a study that investigated 61 antigens [32]. With the nanostring platform, however, 82 RNA targets could be detected in addition to 32 proteins.

5. Conclusions

In this study, we could demonstrate the value of implementing spatial statistics in the analysis of immunofluorescence-stained cancer specimens. This approach enables an objective investigator-independent in situ analysis. We could show that CTL objectively and systemically avoid hypoxic regions within cancers of the head and neck.
It is our vision that the semantics of spatial analysis in human tumor tissue, which is beginning to be established, will serve as building blocks for the spatial interpretation in the setting of far more complex investigations with high multiplexing in the future. To ease implementation of spatial statistical tools for other researchers C.A.N.B. and J.K. developed the Shiny apps mentioned above to upload single-cell data in *.csv-format for exemplary analysis (Figure 3D). The apps can be accessed here (http://apps.math.aau.dk/spatstat/) (accessed on 13 April 2021). We hope that these introductory apps may motivate researchers to use spatial statistical tools to underline statements made from in vivo observations. Such an approach may significantly deepen the pathophysiological understanding of the tumor microenvironment and serve as an in vivo benchmark for the pathophysiological concepts behind novel approaches to cancer treatment.

Supplementary Materials

The following are available online at https://www.mdpi.com/article/10.3390/cancers13081924/s1, Figure S1: Comparison of original scanned image vs. adjusted image, Figure S2 Environment of ROIs, Figure S3: K-functions, Figure S4: pcf, Figure S5: Cross-functions, Table S1: Antibodies (AB), IHC protocol and resulting staining patterns.

Author Contributions

Conceptualization, J.K. and A.M.; methodology, J.K. and A.M.; software, J.K., P.B., E.R., and C.A.N.B.; validation, J.K., A.M., E.R., C.A.N.B., and S.Z.; formal analysis, J.K. and A.M.; investigation, J.K. and A.M.; resources, H.S., A.M., and S.Z. (Biobank); data curation, J.K. and A.M.; writing—original draft preparation, J.K. and A.M.; writing—review and editing, A.M., C.A.N.B., and E.R.; visualization, A.M. and J.K.; supervision, H.S. and A.M.; project administration, H.S. and A.M.; funding acquisition, H.S. and A.M. All authors have read and agreed to the published version of the manuscript.

Funding

This research was partially funded by a grant from VARIAN, Inc.

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki. The use of head and neck cancer tissue specimens was approved by the Ethics Committee of the “Landesärztekammer Rheinland-Pfalz”, approval number 2021-15672.

Informed Consent Statement

Not applicable. Use of surplus tissue specimens is covered by the approval of the local ethics committee. No patient data were analyzed in this study.

Data Availability Statement

Most datasets can be viewed in the app of C.A.N.B. and J.K. A total of 6 datasets from TMA-spots were excluded from the app for technical reasons (round vs. rectangular observation windows). Excluded datasets can be requested from the corresponding author. All calculated function graphs are depicted in the Supplementary Figures S3–S5.

Acknowledgments

Some of the results are part of the doctoral thesis of J.K.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Vaupel, P.; Mayer, A. Hypoxia in cancer: Significance and impact on clinical outcome. Cancer Metastasis Rev. 2007, 26, 225–239. [Google Scholar] [CrossRef] [PubMed]
  2. Nordsmark, M.; Bentzen, S.M.; Rudat, V.; Brizel, D.; Lartigau, E.; Stadler, P.; Becker, A.; Adam, M.; Molls, M.; Dunst, J.; et al. Prognostic value of tumor oxygenation in 397 head and neck tumors after primary radiation therapy. An international multi-center study. Radiother. Oncol. 2005, 77, 18–24. [Google Scholar] [CrossRef] [PubMed]
  3. Hall, J.E.; Giaccia, J.A. Radiobiology for the Radiologist, 8th ed.; Wolters Kluwer VII: Philadelphia, PA, USA, 2019. [Google Scholar]
  4. Walsh, J.C.; Lebedev, A.; Aten, E.; Madsen, K.; Marciano, L.; Kolb, H.C. The Clinical Importance of Assessing Tumor Hypoxia: Relationship of Tumor Hypoxia to Prognosis and Therapeutic Opportunities. Antioxid. Redox Signal. 2014, 21, 1516–1554. [Google Scholar] [CrossRef] [PubMed]
  5. Kao, H.; Lou, P. Immune checkpoint inhibitors for head and neck squamous cell carcinoma: Current landscape and future directions. Head Neck 2019, 41, 4–18. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  6. Spector, M.E.; Bellile, E.; Amlani, L.; Zarins, K.; Smith, J.; Brenner, J.C.; Rozek, L.; Nguyen, A.; Thomas, D.; McHugh, J.B.; et al. Prognostic Value of Tumor-Infiltrating Lymphocytes in Head and Neck Squamous Cell Carcinoma. JAMA Otolaryngol. Neck Surg. 2019, 145, 1012–1019. [Google Scholar] [CrossRef] [PubMed]
  7. Hegde, P.S.; Karanikas, V.; Evers, S. The Where, the When, and the How of Immune Monitoring for Cancer Immunotherapies in the Era of Checkpoint Inhibition. Clin. Cancer Res. 2016, 22, 1865–1874. [Google Scholar] [CrossRef] [Green Version]
  8. Hatfield, S.M.; Kjaergaard, J.; Lukashev, D.; Schreiber, T.H.; Belikoff, B.; Abbott, R.; Sethumadhavan, S.; Philbrook, P.; Ko, K.; Cannici, R.; et al. Immunological mechanisms of the antitumor effects of supplemental oxygenation. Sci. Transl. Med. 2015, 7, 277ra30. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  9. Hatfield, S.M.; Kjaergaard, J.; Lukashev, D.; Belikoff, B.; Schreiber, T.H.; Sethumadhavan, S.; Abbott, R.; Philbrook, P.; Thayer, M.; Shujia, D.; et al. Systemic oxygenation weakens the hypoxia and hypoxia inducible factor 1α-dependent and extracellular adenosine-mediated tumor protection. J. Mol. Med. 2014, 92, 1283–1292. [Google Scholar] [CrossRef]
  10. Baddeley, A.J.; Rubak, E.; Turner, R. Replicated point patterns and designed experiments. In Spatial Point Patterns: Methodology and Applications; Keiding, R.N., Morgan, B.J.T., Wikle, C.K., van der Heijden, P., Eds.; CRC Press, Taylor & Francis Group: Boca Raton, CA, USA; London, UK; New York, NY, USA, 2016; pp. 661–708. [Google Scholar]
  11. Bankhead, P.; Loughrey, M.B.; Fernández, J.A.; Dombrowski, Y.; McArt, D.G.; Dunne, P.D.; McQuaid, S.; Gray, R.T.; Murray, L.J.; Coleman, H.G.; et al. QuPath: Open source software for digital pathology image analysis. Sci. Rep. 2017, 7, 1–7. [Google Scholar] [CrossRef] [Green Version]
  12. Mayer, A.; Zahnreich, S.; Brieger, J.; Vaupel, P.; Schmidberger, H. Downregulation of EGFR in hypoxic, diffusion-limited areas of squamous cell carcinomas of the head and neck. Br. J. Cancer 2016, 115, 1351–1358. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  13. Peranzoni, E.; Lemoine, J.; Vimeux, L.; Feuillet, V.; Barrin, S.; Kantari-Mimoun, C.; Bercovici, N.; Guérin, M.; Biton, J.; Ouakrim, H.; et al. Macrophages impede CD8 T cells from reaching tumor cells and limit the efficacy of anti–PD-1 treatment. Proc. Natl. Acad. Sci. USA 2018, 115, E4041–E4050. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  14. Greig-Smith, P. The Use of Random and Contiguous Quadrats in the Study of the Structure of Plant Communities. Ann. Bot. 1952, 16, 293–316. [Google Scholar] [CrossRef]
  15. Torquato, S.; Lu, B.; Rubinstein, J. Nearest-neighbour distribution function for systems on interacting particles. J. Phys. A Math. Gen. 1990, 23, L103–L107. [Google Scholar] [CrossRef]
  16. Ripley, B.D. Modelling Spatial Patterns. J. R. Stat. Soc. Ser. B Methodol. 1977, 39, 172–192. [Google Scholar] [CrossRef]
  17. Glass, L.; Tobler, W.R.; Glass, W.R.T.L. General: Uniform Distribution of Objects in a Homogeneous Field: Cities on a Plain. Nat. Cell Biol. 1971, 233, 67–68. [Google Scholar] [CrossRef]
  18. Illian, J.; Penttinen, A.; Stoyan, H.; Stoyan, D. Statistical Analysis and Modelling of Spatial Point Patterns; Statistics in Practice; John Wiley: Chichester, UK; Hoboken, NJ, USA, 2008; Volume XIX, 534p. [Google Scholar]
  19. Baddeley, A.J.; Møller, J.; Waagepetersen, R. Non and semi-parametric estimation of interaction in inhomogeneous point patterns. Stat. Neerl. 2001, 54, 329–350. [Google Scholar] [CrossRef]
  20. Harkness, R.D.; Isham, V. A Bivariate Spatial Point Pattern of Ants’ Nests. Appl. Stat. 1983, 32, 293. [Google Scholar] [CrossRef]
  21. Lotwick, H.W.; Silverman, B.W. Methods for Analysing Spatial Processes of Several Types of Points. J. R. Stat. Soc. Ser. 1982, 44, 406–413. [Google Scholar] [CrossRef]
  22. Kaanders, J.H.; Wijffels, K.I.; Marres, H.A.M.; Ljungkvist, A.S.E.; Pop, L.A.M.; van den Hoogen, F.J.; De Wilde, P.C.M.; Bussink, J.; Raleigh, J.A.; Van Der Kogel, A.J. Pimonidazole binding and tumor vascularity predict for treatment outcome in head and neck cancer. Cancer Res. 2002, 62, 7066–7074. [Google Scholar]
  23. Rafajová, M.; Zatovicová, M.; Kettmann, R.; Pastorek, J.; Pastoreková, S. Induction by hypoxia combined with low glucose or low bicarbonate and high posttranslational stability upon reoxygenation contribute to carbonic anhydrase IX expression in cancer cells. Int. J. Oncol. 2004, 24, 995–1004. [Google Scholar] [CrossRef]
  24. Buckle, C.E.; Udawatta, V.; Straus, C.M. Now You See It, Now You Don’t: Visual Illusions in Radiology. RadioGraphics 2013, 33, 2087–2102. [Google Scholar] [CrossRef]
  25. Albright, T.D. High-Level visual processing: Cognitive influences. In Principles of Neural Science; Kandel, E.R., Schwartz, J.H., Jessell, T.M., Siegelbuam, S.A., Hudspeth, A.J., Eds.; McGraw-Hill: New York City, NY, USA, 2013; pp. 621–637. [Google Scholar]
  26. Harmon-Jones, E. A cognitive dissonance theory perspective on persuasion. In The Persuasion Handbook: Developments in Theory and Practice; SAGE Publications, Inc.: Thousand Oaks, CA, USA, 2002; pp. 99–116. [Google Scholar]
  27. Carstens, J.L.; De Sampaio, P.C.; Yang, D.; Barua, S.; Wang, H.; Rao, A.; Allison, J.P.; LeBleu, V.S.; Kalluri, R. Spatial computation of intratumoral T cells correlates with survival of patients with pancreatic cancer. Nat. Commun. 2017, 8, 15095. [Google Scholar] [CrossRef]
  28. Toki, M.I.; Merritt, C.R.; Wong, P.F.; Smithy, J.W.; Kluger, H.M.; Syrigos, K.N.; Ong, G.T.; Warren, S.E.; Beechem, J.M.; Rimm, D.L. High-Plex Predictive Marker Discovery for Melanoma Immunotherapy-Treated Patients Using Digital Spatial Profiling. Clin. Cancer Res. 2019, 25, 5503–5512. [Google Scholar] [CrossRef]
  29. Jackson, H.W.; Fischer, J.R.; Zanotelli, V.R.T.; Ali, H.R.; Mechera, R.; Soysal, S.D.; Moch, H.; Muenst, S.; Varga, Z.; Weber, W.P.; et al. The single-cell pathology landscape of breast cancer. Nat. Cell Biol. 2020, 578, 615–620. [Google Scholar] [CrossRef] [PubMed]
  30. Keren, L.; Bosse, M.; Marquez, D.; Angoshtari, R.; Jain, S.; Varma, S.; Yang, S.-R.; Kurian, A.; Van Valen, D.; West, R.; et al. A Structured Tumor-Immune Microenvironment in Triple Negative Breast Cancer Revealed by Multiplexed Ion Beam Imaging. Cell 2018, 174, 1373–1387.e19. [Google Scholar] [CrossRef] [Green Version]
  31. Yan, Y.; Leontovich, A.A.; Gerdes, M.J.; Desai, K.; Dong, J.; Sood, A.; Santamaria-Pang, A.; Mansfield, A.S.; Chadwick, C.; Zhang, R.; et al. Understanding heterogeneous tumor microenvironment in metastatic melanoma. PLoS ONE 2019, 14, e0216485. [Google Scholar] [CrossRef] [Green Version]
  32. Gerdes, M.J.; Sevinsky, C.J.; Sood, A.; Adak, S.; Bello, M.O.; Bordwell, A.; Can, A.; Corwin, A.; Dinn, S.; Filkins, R.J.; et al. Highly multiplexed single-cell analysis of formalin-fixed, paraffin-embedded cancer tissue. Proc. Natl. Acad. Sci. USA 2013, 110, 11982–11987. [Google Scholar] [CrossRef] [PubMed] [Green Version]
Figure 1. A representative area from a whole-slide specimen is shown to demonstrate the procedure of data acquisition in QuPath, cell classification, and transfer to R. (A) Tumor section after single-cell detection in QuPath. Approximated cell borders, extrapolated from the positions of cell nuclei, which had been segmented based on the DAPI channel (blue), are depicted by red lines. CD8-positive cytotoxic T lymphocytes (CTL) are shown in yellow. Hypoxic, CA IX-positive tumor cells are shown in green. (B) Result of the tumor-stroma classification using QuPath’s machine learning features. A set of example areas for tumor and stroma were delineated by the user. QuPath was then able to discriminate between both compartments for the rest of the cells in the specimen. (C,D) Construction of additional single-threshold classifiers for hypoxic cells (C) and CD8-positive CTL (D). (E) A compound classifier comprising tumor, stroma, and subclassifications according to the single-measurement classifiers shown in panels (C,D). Scale bar in (E) applies to panels AE. (F) A color-coded plot of cell positions generated in R after transfer of the data and conversion to a multitype planar point pattern (ppp) in Spatstat. For the sake of clarity, subclassifications have been reorganized to CD8-positive CTL (yellow), normoxic tumor cells (brown), and hypoxic tumor cells (green). Original subclassifications are shown in panel (G).
Figure 1. A representative area from a whole-slide specimen is shown to demonstrate the procedure of data acquisition in QuPath, cell classification, and transfer to R. (A) Tumor section after single-cell detection in QuPath. Approximated cell borders, extrapolated from the positions of cell nuclei, which had been segmented based on the DAPI channel (blue), are depicted by red lines. CD8-positive cytotoxic T lymphocytes (CTL) are shown in yellow. Hypoxic, CA IX-positive tumor cells are shown in green. (B) Result of the tumor-stroma classification using QuPath’s machine learning features. A set of example areas for tumor and stroma were delineated by the user. QuPath was then able to discriminate between both compartments for the rest of the cells in the specimen. (C,D) Construction of additional single-threshold classifiers for hypoxic cells (C) and CD8-positive CTL (D). (E) A compound classifier comprising tumor, stroma, and subclassifications according to the single-measurement classifiers shown in panels (C,D). Scale bar in (E) applies to panels AE. (F) A color-coded plot of cell positions generated in R after transfer of the data and conversion to a multitype planar point pattern (ppp) in Spatstat. For the sake of clarity, subclassifications have been reorganized to CD8-positive CTL (yellow), normoxic tumor cells (brown), and hypoxic tumor cells (green). Original subclassifications are shown in panel (G).
Cancers 13 01924 g001
Figure 2. (A) The most simple method to assess clustering of cells is the quadrat test. Left panel: distribution of CTL (yellow) in the same region of a squamous cell carcinoma of the head and neck depicted in Figure 1. Clustering is less evident visually when CTL are viewed separately, outside of the context of tumor and stroma areas. First, the quadrat test (right panel) shows statistically significant inhomogeneity (p < 2.2 × 10−16). The numbers within the quadrats (from left to right) represent the number of events in each quadrat, the average number in all quadrats, and the residuals. (B) The G-function is another method to assess the clustering of CTL. Left panel: Graph of nearest neighbor distances of CTL (yellow lines), depicted as an overlay on the same tumor area. Right panel: a plot of the G-function (for any given CTL) shows a higher probability of the presence of the nearest neighboring CTL within a distance of approximately 8–23 µm than would be expected in a random distribution. The low values in the distances leading up to 8 µm are a result of the technical shrinkage of cells with a certain diameter to points for the sake of the calculation. (C) For the K-function, the average number of neighboring cell events within a circle of a given radius r is calculated for each cell event in the population. Left panel: representative circles of r = 50 µm are shown. This process is repeated for every value of r up to 100 µm in the region of interest (ROI). K-functions are standardized, and corrections for edge effects are performed, the details of which have been explained before. The heterogeneity of cell event numbers within these circles is evident. Right panel: a plot of the K-function (orange), corrected for inhomogeneity with its corresponding 95%-confidence band (green and yellow lines). Although an upward deviation from the Poisson distribution (complete spatial randomness (CSR), brown line) is evident, this did not leave the 95% confidence bands. Hence, the deviation from CSR in this example using the K-function is not statistically significant (D). The pair correlation function (pcf) of the same dataset shows pronounced clustering at low distances. Left panel: contrary to the K-function, the pcf analyzes the number of cells on expanding rings, thereby scanning through layers around each individual CTL. Right panel: Contrary to the K-function, the pcf shows significant clustering in a band between 7 and 15 µm around the cell, whereas no deviation from CSR is evident at distances beyond.
Figure 2. (A) The most simple method to assess clustering of cells is the quadrat test. Left panel: distribution of CTL (yellow) in the same region of a squamous cell carcinoma of the head and neck depicted in Figure 1. Clustering is less evident visually when CTL are viewed separately, outside of the context of tumor and stroma areas. First, the quadrat test (right panel) shows statistically significant inhomogeneity (p < 2.2 × 10−16). The numbers within the quadrats (from left to right) represent the number of events in each quadrat, the average number in all quadrats, and the residuals. (B) The G-function is another method to assess the clustering of CTL. Left panel: Graph of nearest neighbor distances of CTL (yellow lines), depicted as an overlay on the same tumor area. Right panel: a plot of the G-function (for any given CTL) shows a higher probability of the presence of the nearest neighboring CTL within a distance of approximately 8–23 µm than would be expected in a random distribution. The low values in the distances leading up to 8 µm are a result of the technical shrinkage of cells with a certain diameter to points for the sake of the calculation. (C) For the K-function, the average number of neighboring cell events within a circle of a given radius r is calculated for each cell event in the population. Left panel: representative circles of r = 50 µm are shown. This process is repeated for every value of r up to 100 µm in the region of interest (ROI). K-functions are standardized, and corrections for edge effects are performed, the details of which have been explained before. The heterogeneity of cell event numbers within these circles is evident. Right panel: a plot of the K-function (orange), corrected for inhomogeneity with its corresponding 95%-confidence band (green and yellow lines). Although an upward deviation from the Poisson distribution (complete spatial randomness (CSR), brown line) is evident, this did not leave the 95% confidence bands. Hence, the deviation from CSR in this example using the K-function is not statistically significant (D). The pair correlation function (pcf) of the same dataset shows pronounced clustering at low distances. Left panel: contrary to the K-function, the pcf analyzes the number of cells on expanding rings, thereby scanning through layers around each individual CTL. Right panel: Contrary to the K-function, the pcf shows significant clustering in a band between 7 and 15 µm around the cell, whereas no deviation from CSR is evident at distances beyond.
Cancers 13 01924 g002
Figure 3. Exemplary pattern distributions and the corresponding K-function and pcf. All images in this figure were taken from the Shiny apps by C.A.N.B. and J.K. Lines corresponding to the computed functions are shown in red, while the lines corresponding to a Poisson distribution are shown in blue. (A) A randomly distributed pattern (Poisson distribution). While the pcf in the second column varies around the line of a theoretical Poisson distribution, the K-function of the pattern in the third column is very close to the line of a theoretical Poisson distribution, implying that the pattern might be close to complete spatial randomness (CSR). (B) A pattern with a clustering behavior. The corresponding pcf in the second column stays mostly above the pcf of a theoretical Poisson distribution, as does the K-function in the third column. This implies a clustering behavior. (C) A pattern with inhibitive behavior. The corresponding pcf in the second column stays mostly below the pcf of CSR. The K-function in the third column is below the line of CSR as well, which implies an inhibitive or regular behavior. (D) An exemplary dataset of the second app where tumor (blue points) and stroma (red points) cells were differentiated. Instead of the K-function and pcf used before, their Cross-functions are being used. Here one can clearly see an inhibitive or avoiding pattern where cells of one type are mostly close to each other while avoiding close contact with cells of the other type. This effect is enhanced by the inhomogeneity in the general distribution, with tumor cells being mainly on the left side of the ROI and stromal cells being almost exclusively on the right half.
Figure 3. Exemplary pattern distributions and the corresponding K-function and pcf. All images in this figure were taken from the Shiny apps by C.A.N.B. and J.K. Lines corresponding to the computed functions are shown in red, while the lines corresponding to a Poisson distribution are shown in blue. (A) A randomly distributed pattern (Poisson distribution). While the pcf in the second column varies around the line of a theoretical Poisson distribution, the K-function of the pattern in the third column is very close to the line of a theoretical Poisson distribution, implying that the pattern might be close to complete spatial randomness (CSR). (B) A pattern with a clustering behavior. The corresponding pcf in the second column stays mostly above the pcf of a theoretical Poisson distribution, as does the K-function in the third column. This implies a clustering behavior. (C) A pattern with inhibitive behavior. The corresponding pcf in the second column stays mostly below the pcf of CSR. The K-function in the third column is below the line of CSR as well, which implies an inhibitive or regular behavior. (D) An exemplary dataset of the second app where tumor (blue points) and stroma (red points) cells were differentiated. Instead of the K-function and pcf used before, their Cross-functions are being used. Here one can clearly see an inhibitive or avoiding pattern where cells of one type are mostly close to each other while avoiding close contact with cells of the other type. This effect is enhanced by the inhomogeneity in the general distribution, with tumor cells being mainly on the left side of the ROI and stromal cells being almost exclusively on the right half.
Cancers 13 01924 g003
Figure 4. Demonstration of the repellent effect of hypoxic tissue areas on CTL infiltration. Six examples (AF) showing significantly stronger inhibitory effects of hypoxic tumor cells compared to normoxic tumor cells concerning the distribution of CTL. Each row corresponds to one tumor. Left panels: original immunofluorescence images (DAPI, blue; hypoxic cells, green; CTL, yellow), all scale bars: 200 µm. Left middle panels, corresponding Spatstat point pattern representation of the different groups of cells identified in the tumor sections (hypoxic tumor cells, green; normoxic tumor cells, dark red; CTL, yellow). Right middle panels, inhomogeneity-corrected Kcross functions testing the distribution of CTL vs. hypoxic (green) and normoxic (red) tumor cells. The areas shown represent the confidence bands generated by Loh’s bootstrap method. Right panels, plot illustrating the delta between the lowest value of the red curve vs. the highest values of the green curves in the corresponding right middle panels. Values below 0 define a significant deviation. This is the region of statistically significant divergence between the distribution of both tumor cell populations relative to CTL. It is evident that hypoxic tumor cells have a stronger repellent or anti-clustering effect on CTL than normoxic tumor cells.
Figure 4. Demonstration of the repellent effect of hypoxic tissue areas on CTL infiltration. Six examples (AF) showing significantly stronger inhibitory effects of hypoxic tumor cells compared to normoxic tumor cells concerning the distribution of CTL. Each row corresponds to one tumor. Left panels: original immunofluorescence images (DAPI, blue; hypoxic cells, green; CTL, yellow), all scale bars: 200 µm. Left middle panels, corresponding Spatstat point pattern representation of the different groups of cells identified in the tumor sections (hypoxic tumor cells, green; normoxic tumor cells, dark red; CTL, yellow). Right middle panels, inhomogeneity-corrected Kcross functions testing the distribution of CTL vs. hypoxic (green) and normoxic (red) tumor cells. The areas shown represent the confidence bands generated by Loh’s bootstrap method. Right panels, plot illustrating the delta between the lowest value of the red curve vs. the highest values of the green curves in the corresponding right middle panels. Values below 0 define a significant deviation. This is the region of statistically significant divergence between the distribution of both tumor cell populations relative to CTL. It is evident that hypoxic tumor cells have a stronger repellent or anti-clustering effect on CTL than normoxic tumor cells.
Cancers 13 01924 g004
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Kaufmann, J.; Biscio, C.A.N.; Bankhead, P.; Zimmer, S.; Schmidberger, H.; Rubak, E.; Mayer, A. Using the R Package Spatstat to Assess Inhibitory Effects of Microregional Hypoxia on the Infiltration of Cancers of the Head and Neck Region by Cytotoxic T Lymphocytes. Cancers 2021, 13, 1924. https://doi.org/10.3390/cancers13081924

AMA Style

Kaufmann J, Biscio CAN, Bankhead P, Zimmer S, Schmidberger H, Rubak E, Mayer A. Using the R Package Spatstat to Assess Inhibitory Effects of Microregional Hypoxia on the Infiltration of Cancers of the Head and Neck Region by Cytotoxic T Lymphocytes. Cancers. 2021; 13(8):1924. https://doi.org/10.3390/cancers13081924

Chicago/Turabian Style

Kaufmann, Justus, Christophe A. N. Biscio, Peter Bankhead, Stefanie Zimmer, Heinz Schmidberger, Ege Rubak, and Arnulf Mayer. 2021. "Using the R Package Spatstat to Assess Inhibitory Effects of Microregional Hypoxia on the Infiltration of Cancers of the Head and Neck Region by Cytotoxic T Lymphocytes" Cancers 13, no. 8: 1924. https://doi.org/10.3390/cancers13081924

APA Style

Kaufmann, J., Biscio, C. A. N., Bankhead, P., Zimmer, S., Schmidberger, H., Rubak, E., & Mayer, A. (2021). Using the R Package Spatstat to Assess Inhibitory Effects of Microregional Hypoxia on the Infiltration of Cancers of the Head and Neck Region by Cytotoxic T Lymphocytes. Cancers, 13(8), 1924. https://doi.org/10.3390/cancers13081924

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop