Rapidly Quantifying Interior Greenery Using 360° Panoramic Images

Abstract

Many people spend the majority of their time indoors and there is emerging evidence that interior greenery contributes to human wellbeing. Accurately capturing the amount of interior greenery is an important first step in studying its contribution to human well-being. In this study, we evaluated the accuracy of interior greenery captured using 360° panoramic images taken within a range of different interior spaces. We developed an Interior Green View Index (iGVI) based on a K-means clustering algorithm to estimate interior greenery from 360° panoramic images taken within 66 interior spaces and compared these estimates with interior greenery measured manually from the same panoramic images. Interior greenery estimated using the automated method ranged from 0% to 34.19% of image pixels within the sampled interior spaces. Interior greenery estimated using the automated method was highly correlated (r = 0.99) with interior greenery measured manually, although we found the accuracy of the automated method compared with the manual method declined with the volume and illuminance of interior spaces. The results suggested that our automated method for extracting interior greenery from 360° panoramic images is a useful tool for rapidly estimating interior greenery in all but very large and highly illuminated interior spaces.

1. Introduction

Contact with green space has positive effects on physical and mental health [1,2,3]. Some people (e.g., with limited mobility, living in densely-populated cities, and living in extreme climates) spend the majority of their time indoors [4,5,6,7,8]. For these people, a large proportion of their contact with green space may occur indoors [9,10]. Interior green space (Figure 1) can provide some of the environmental and psychological benefits provided by outdoor green space [11]. For example, interior green space is associated with greater parasympathetic activity (i.e., rest) [12], attention span [13,14], and improved air quality [15,16]. Thus, interior green space has potential to fulfil some of the functions of outdoor green space for those with limited opportunities to access the latter. However, the benefits of interior green space have received relatively little attention compared with the benefits of outdoor green space [17,18].

Vegetation (or greenery) is typically measured using some form of remote sensing. With continual improvement in spatial resolution, airborne remote sensing is the most common method used to capture greenery in outdoor spaces [19]. Airborne remote sensing technology allows large areas to be sampled rapidly in contrast to on-site observation. However, airborne remote sensing instruments are not suitable for capturing indoor greenery. Alternatively, there are a number of hand-held sensors that can potentially be used to capture interior greenery. Terrestrial LiDAR scanners can be used to obtain Leaf Area Density and Leaf Area Index at a high level of accuracy [20]. Previous studies have demonstrated that terrestrial LiDAR can accurately calculate forest structure at the branch level, individual tree level, and plot level [20,21,22]. However, LiDAR instruments are expensive [23] and the processing of large amounts of point cloud data that LiDAR produces can be challenging [24,25]. Using conventional RGB images for capturing greenery is a cost-effective method in some circumstances [26]. Yang et al. [27] used RGB images taken in cardinal directions to evaluate the visibility of greenery at the street level and Yu et al. [28] used RGB images to evaluate greenery in urban parks.

Several methods have been developed to extract indoor greenery from RGB imagery captured using conventional RGB cameras. These are based on calculating the area of greenery [29,30] or the ratio of greenery relative to other features [31]. Choi et al. [11] developed an index of greenness based on the percentage of plant leaf area within the total area captured in RGB images. Salamone et al. [32] introduced an index that indicates the fraction of green area on the surface of a hemisphere centered on the point of analysis using RGB imagery captured with a fisheye lens. However, past methods for capturing interior greenery have involved processing multiple images [33,34].

The use of 360° panoramic images may represent an alternative for capturing the way that humans experience interior greenery. As the name suggests, 360° panoramic images can capture plants in all cardinal, horizontal, and vertical directions in one continuous image from a single, ground-based point, and thus have the potential to be an effective method for measuring interior greenery in a cost-effective way. The aims of this study are therefore to: (a) develop an automated method to extract interior greenery from 360° panoramic RGB images; (b) evaluate the accuracy of this method within interior spaces with different dimensions.

2. Materials and Methods

2.1. Selection of Interior Spaces

In this study, we focused on sampling interior spaces within public buildings used primarily for work or study (i.e., university buildings, offices, and libraries) and shopping, leisure, or accommodation (i.e., museums, shopping centers, and hotels). In order to sample interior spaces with a range of dimensions, we stratified interior spaces into 3 types (i.e., atriums, corridors, and meeting rooms) and sampled an approximately even number of each type of interior spaces. Only interior spaces that contained at least some greenery were sampled (i.e., planted flowers, trees, flower beds, green walls, and indoor gardens). In total, we sampled 66 interior spaces from 33 public buildings. All sampling occurred in June and July 2021 and all buildings were located in the city of Canberra, Australia (35.2809° S, 149.1300° E).

2.2. Panoramic Imagery Captured within Interior Spaces

One 360° panoramic image was captured in the center of each of the 66 sampled interior spaces using an Insta360 One R camera. The dual lens of the camera can capture 360-degrees simultaneously with 5.7 K resolution (Figure 2). Each image was captured with the camera positioned 1.7 m above the ground. 360° panoramic images were generated from Insta360 Studio 2021 software, which is the photo-stitching software provided with the Insta360 One R camera (Figure 2a). Distortions in the upper and lower part of 360° panoramic images can affect the quality of the image [35]. Tsai et al. [36] indicated that only the center section of panoramic images has relatively low distortion. Yin et al. [37] and Ki et al. [38] overcame this by cropping the top and bottom of the image to match a viewer’s perspective. We used the same method (i.e., we cropped the central part of each 360° panoramic image) to simulate the viewer’s perspective (Figure 2b).

2.3. Automated Greenery Extraction from Panoramic Images

Green vegetation extraction from remote sensing images has been studied for almost four decades [39]. Green vegetation can be easily detected using remote sensing because of its high reflectance and high absorption characteristics at the near-infrared and red parts of the electromagnetic spectrum. 360° panoramic images contain no near-infrared bands but red, green, and blue bands. We chose a K-means clustering algorithm to extract greenery from the 360° panoramic images. Images were uploaded to an image processing script tool (Image Color Summarizer) [40], which is an open-source online tool developed by Michael Smith Genome Sciences Centre. The processing time for the K-means clustering algorithm to evaluate 360° panoramic images is under 30 s with very high level of precision (200 px). Finally descriptive color statistics were computed using every pixel in panoramic image.

A K-means clustering algorithm was utilized for the partitioning of images into clusters. Key steps are followed:

• Assign the pixels to each cluster that restrict the partition between the pixel and the cluster center;
• Calculate the color border of the cluster by the average value of its pixels.

The K-means clustering algorithm segments areas of interest from the background using an unsupervised pixel-based classification method [41,42] with high computational efficiency [43].

Calculating the Amount of Interior Greenery from 360° Panoramic Images

Previous approaches for calculating greenery from RGB imagery are based on combining data from multiple images. Yang et al. [27] proposed a Green View Index (GVI) to extract greenery from RGB imagery in urban forests. Their GVI was defined as the ratio of green pixels to the total pixels captured in photos taken in four cardinal directions using the following equation:

$$\mathrm{GVI}=\frac{{{\displaystyle \sum }}_{i=1}^{4}Are{a}_g}{{{\displaystyle \sum }}_{i=1}^{4}Are{a}_t}\times 100$$

(1)

where $Are{a}_g$ is the green pixels of the photo taken in the$i$th cardinal direction, and$Are{a}_t$is all pixels of the photo taken in the $i$th cardinal direction. Dong et al. [44] proposed a GVI to estimate greenery using different horizontal and vertical directions at each sample site as:

$$\mathrm{GVI}=\frac{{{\displaystyle \sum }}_{j=1}^n{{\displaystyle \sum }}_{i=1}^{m}Are{a}_{g\_ij}}{{{\displaystyle \sum }}_{j=1}^n{{\displaystyle \sum }}_{i=1}^{m}Are{a}_{t\_ij}}\times 100$$

(2)

where $Are{a}_{g\_ij}$ is the green pixels of the photo taken in the horizontal direction $i$and the vertical direction $j$ for each sample site, and $Are{a}_{t\_ij}$ is the whole pixels of the photo in the horizontal direction $i$ and the vertical direction$j$. Parameters $m$ and $n$ represent the numbers of horizontal and vertical directions, respectively.

Because a 360° panoramic image represents all cardinal, horizontal and vertical directions in one continuous image, we calculated interior greenery using a simplified Interior Green View Index (iGVI) from 360° panoramic images as:

$$\mathrm{iGVI}=\frac{Are{a}_g}{Are{a}_t}\times 100$$

(3)

where $Are{a}_g$ is the number of pixels representing greenery in each 360° panoramic image, and $Are{a}_t$ is the total pixels of the 360° panoramic image.

2.4. Evaluation of the Interior Green View Index

To evaluate the accuracy of our automated method for extracting interior greenery from 360° panoramic images, we compared our estimates of greenery for each interior space derived from the K-means clustering algorithm and iGVI with reference data extracted manually from the same 360° panoramic images. Adobe Photoshop CC 2020 software was used to extract greenery from the 360° panoramic images manually. We used the quick selection tool from Adobe Photoshop to select the area of greenery in each image. Although we were unable to validate the accuracy of this method, greenery within the imagery is clearly visible and can be captured at a high spatial accuracy given the images are captured at 5.7 K resolution. The number of selected pixels representing the greenery of an image can be read directly from the histogram toolbox in the Adobe Photoshop package. The selected pixels were divided by the total pixels of an image to calculate the iGVI for each 360° panoramic image. This process took up to 4 min for each image. The discrepancy in the percent of greenery estimated for each interior space between the K-means methods, and the manual method was used to evaluate the accuracy of the former.

To examine which variables affect differences between observed values of interior greenery (using the manual extraction method) and predicted values of interior greenery (using the automated extraction method), we measured seven variables (covariates) for each interior space where 360° panoramic images were captured (

Table 1. Variables measured for each interior space where 360° panoramic images were captured.

Variable	Definition
length	length of interior space
width	width of interior space
height	height of interior space
area	total floor area of interior space calculated from its interior surfaces
volume	measurement of a three-dimensional shape that is enclosed by a closed area
illuminance	total luminous flux incident on a surface
type	several specific areas or design settings related to social behavior

). We collected illuminance (lux) at each camera position using the ambient light sensor on a smartphone and Light Meter application [45] and the dimensions (length, width, height, area, and volume) of each sampled interior space using a laser rangefinder. Differences between observed and predicted iGVI values were used as the response variable in 14 linear regression models in which we fit different combinations of the seven covariates (

Table 2. Variables included in the 14 linear regression models that we compared to evaluate the automated method we developed to extract interior greenery.

Model	Variable
1	illuminance
2	width
3	length
4	Area
5	volume
6	Type
7	Illuminance + width
8	Illuminance + width + length
9	Illuminance + width + length + height
10	Illuminance + area
11	illuminance + volume
12	Illuminance + type
13	Illuminance + type + width
14	Illuminance + type + width + length

). We used Akaike’s Information Criterion (AIC_c) adjusted for small sample sizes to rank the 14 models in Table 2 [46]. We calculated Variance Inflation Factors (VIF) for each variable in the top-ranked model as a check for collinearity. All statistical analyses were undertaken using R software [47].

3. Results

We sampled 66 interior spaces from 33 public buildings. We sampled 23 atriums, 21 corridors, and 21 meeting rooms. Interior spaces had lengths, widths, and heights (to the ceiling) that ranged from 5.5 to 92.1 m, 1.7 to 55.4 m, and 2.0 to 20.3 m, respectively. Interior areas ranged from 19 to 3933 m² and interior volumes ranged from 58 to 78,275 m³. Illuminances ranged from 1 to 1632 lux.

3.1. Interior Greenery Estimates

Interior greenery was estimated for each of the 66 interior spaces using the manual and automated methods. Figure 3 illustrates a 360° panoramic image for one interior space (Figure 3a), greenery extracted using the automated method (Figure 3b), and greenery extracted using the manual method (Figure 3c). The estimated percent greenery for the 66 interior spaces extracted using the automated method ranged from 0% to 34.19% cover (mean = 2.3%).

3.2. Differences between Observed and Predicted Greenery for Interior Spaces

All of the 66 360° panoramic images were used to analyze the automated extraction results using K-means clustering algorithm. There was a strong correlation (r = 0.99, Pearson correlation coefficient) between the amount of greenery estimated for the 66 interior spaces manually and the amount of greenery estimated for these interior spaces using the automated method (Figure 4). However, there were some differences between greenery estimated using the manual and automated methods (Figure 5).

From the fourteen linear regression models we fitted, Model 11, which contained illuminance and interior volume, received the lowest AIC value, indicating the strongest empirical support. This model had an AIC weight of 0.19, indicating there was a 19% chance this was the strongest model selected. However, a further four models had empirical support (i.e., ∆AICc ≤ 2) (

Table 3. Linear regression models with empirical support (i.e., ∆AICc ≤ 2) that were used to predict differences in greenery for interior spaces estimated using manual and automated methods. A “+” symbol indicates the variable was included in the model.

Model	Width	Length	Area	Volume	Illuminance	∆AICc
11				+	+	0
5				+		0.3976
4			+			0.7893
10			+		+	1.0351
8	+	+			+	1.7755

). Predictions from Model 11 indicated that the automated method was more likely to over-predict greenery within interior spaces with larger volume and higher illuminance, although there is greater uncertainty in our predictions (i.e., wider confidence limits) for interior spaces with very large volumes and very high illuminance because of a small number of samples at extreme values for these variables (Figure 6).

4. Discussion

The aims of this study were to develop an automated method to extract interior greenery from 360° panoramic images and evaluate the accuracy of this method. Our results indicate that 360° panoramic images can be used to estimate greenery within interior spaces spanning a range of dimensions with a high level of accuracy.

4.1. The Accuracy of Extraction Results

Our results indicated that there is a strong correlation (r = 0.99) between interior greenery estimated manually and interior greener estimated using a K-means clustering algorithm. However, our results indicated that the accuracy of the automated method declines with the volume and illuminance of interior spaces. This could be because the distance between the camera and greenery affects boundary pixels of greenery during image processing [48], which suggests that the noise increases due to the color mixing of the greenery boundary.

We found that interior spaces with higher illuminance cause greater differences between the two methods. Previous studies [49] have shown that high illuminance can change the appearance of colors in the image-capturing stage. Such color constancy problems in images might relate to shadows, indoor artificial lights, and direct sunlight. However, K-means image segmentation, which is a simple automatically unsupervised classification, tends to lack robustness to illuminance changes [50]. Therefore, when assessing interior greenery with this method, researchers should choose a cloudy day and switch lights off to capture the 360° panoramic images in order to reduce the influence caused by sharp contrasts in illuminance.

4.2. Strengths and Limitations

There are several strengths of our research. First, we focus on the assessment of greenery in interior space from a pedestrian’s perspective, which is not the case for most studies of urban green space [27,38,51]. We established a reliable interior green view index, which could be a comparable parameter measured in future studies with similar aims. Second, it is efficient and affordable to use 360° panoramic images to quantify interior greenery compared with some other methods [52,53,54]. Other conventional electronic devices, such as digital cameras, tablets, and smartphones, usually overlap multiple rectangle images into one panoramic image. This easily produces visible seams due to changes in scene illumination of single rectangle images or spatial alignment errors [55]. The Insta360 One R camera we used makes it less challenging to blend images with seamless edges. Moreover, the K-means clustering algorithm we used to extract greenery from the images was relatively simple and efficient. Considering that artificial green features are hard to distinguish, deep learning algorithms based on a greater number of training datasets could be included as part of the interior greenery segmentation process [56].

However, there are still several concerns that need to be addressed. A 360° panoramic image does not capture greenery from all sites in an interior space. That is, greenery captured by a 360° panoramic image is limited to the greenery that can be seen from a central point. The second issue is the absence of near-infrared bands on RGB images [51]. Different plant species, which can have different effects on wellbeing, cannot be distinguished using RGB imagery. For example, the amount of volatile organic compounds removed by interior greenery varies widely by plant species [57]. These issues could be overcome by capturing interior greenery using near-infrared sensors [58], although these are more expensive than a panoramic camera. Third, we captured 360° panoramic images at a specific time rather than over a period of time. However, this may not be an issue when capturing interior vegetation given its spectral properties are less variables than outdoor vegetation [32].

4.3. Future Directions

Our method for capturing interior greenery has applications for assessing and monitoring greenery within interior spaces. There is increasing interest in how the use of interior greenery may influence human wellbeing [52,59]. Many previous studies have focused on counting the number of plants or leaf area, which does not consider the relationship between the interior greenery and space [60]. Our approach can be used as a tool to help researchers and others to quantify interior greenery accurately and cost-effectively, which is an important first step for exploring associations between interior greenery and human wellbeing.

5. Conclusions

In this study we used 360° panoramic images with a newly proposed iGVI, which are closer to the viewer’s perspective, to assess greenery in interior spaces and compared an automated method for estimating greenery with a manual method. The estimated percent greenery for the 66 interior spaces extracted using the automated method ranged from 0% to 34.2% (mean = 2.3%). There was a high correlation (r = 0.99) between the amount of greenery estimated manually and the amount of greenery estimated for these interior spaces using the automated method. The results indicated that 360° panoramic images classified using an automated method based on the K-means clustering algorithm, can capture interior greenery efficiently and accurately which, in turn, could be employed to monitor or research interior greenery and its association with human wellbeing.