A Computer Vision Milky Way Compass

Abstract

The Milky Way is used by nocturnal flying and walking insects for maintaining heading while navigating. In this study, we have explored the feasibility of the method for machine vision systems on autonomous vehicles by measuring the visual features and characteristics of the Milky Way. We also consider the conditions under which the Milky Way is used by insects and the sensory systems that support their detection of the Milky Way. Using a combination of simulated and real Milky Way imagery, we demonstrate that appropriate computer vision methods are capable of reliably and accurately extracting the orientation of the Milky Way under an unobstructed night sky. The technique presented achieves angular accuracy of better then $\pm 2$° under moderate light pollution conditions but also demonstrates that higher light pollution levels will adversely effect orientation estimates by systems depending on the Milky Way for navigation.

1. Introduction

Locomotion in straight lines allows organisms to forage, migrate and escape from competition or danger; it is fundamental to the survival of many species. Orientation is a fundamental operation for closed-loop control that allows for effective locomotion. To successfully find the right direction, some animals use landmarks, magnetic cues or wind, while some species use the Sun, the Moon or polarised light as the cue for orientation [1,2,3]. Visual cues from the sky dome are believed to have been used for synchronisation and orientation by organisms almost from the dawn of life, when the first photosensitive cells formed [4]. Navigation using the stars has been undertaken by humans for millennia and has been found in animals such as seals with sophisticated brains and high resolution eyes [5]. Amongst these celestial options for orientation is the Milky Way, although its characteristics for navigation are different from the pinpoint precision offered by stars for navigation and are more favourable for low angular resolution visual systems.

1.1. The Milky Way

The Milky Way (MW) is a large structure compared to other celestial bodies; a typical image of it is shown in Figure 1. It is an irregular luminous band as much as 30° wide, composed of stars and gas clouds stretching across the night sky visible by an observer on Earth (see Figure 2). Astronomers have known since the 1920s that this band is an edge-on view of the MW Galaxy in which we live, amongst a vast pinwheel teeming with nebulae, gas clouds and hundreds of billions of stars in a class of galaxy known as spiral galaxies. The MW Galaxy is approximately 87,400 light years in diameter and is only 1000 light years across in the spiral arms. The Sun, Earth’s star, is located in the Orion spar about 28,000 light years from the galactic centre. There is a visible structure of the MW seen as dark bands and areas within a luminous band caused by interstellar clouds in cosmic dust, as shown in Appendix A Figure A1, collectively known as the Great Rift or Dark Rift. The luminosity of the MW is not constant along its arc, thus the luminosity of the visible portion of the MW will depend on the observer’s location on Earth and the time of year.

For sensing purposes, the MW is comparatively low contrast; however, it is much larger than the pinpoint stars, and its primary features are composed of low spatial frequencies. It is apparent that sensitivity to low contrast and low light is required to observe it; however, high spatial acuity is not required. This is discussed in the context of insect vision in the next section.

1.2. Insect Vision under Starlight

The method we present is based on observed insect behaviour; thus, it is important to understand what they can actually sense and the limitations of their sensing, since their visual perception is substantially different from our own. Insects have evolved compound eyes that have extraordinary adaptations and specialisations [6], and the tiny optics and eyes of insects are challenged by low light levels. There are two main types of compound eyes, as shown in Figure 3, which are apposition eyes and superposition eyes. In an apposition eye, each ommatidium is sleeved by light-absorbing screening pigment, preventing light reaching the photoreceptors from neighbouring ommatidia. This means that a photoreceptor only receives light from a single lens. Apposition compound eyes are typical of diurnal insects that have evolved to achieve the highest possible angular resolution, such as dragonflies [7] and honey bees [8]. In superposition compound eyes, however, there is a wide optically transparent region, the clear zone (marked cz in Figure 3), between the lenses and the retina. The superposition arrangement allows light from possibly hundreds of lenses to be focused onto photoreceptors in the retina, which improves light sensitivity at the expense of angular resolution. This eye type is more suited to nocturnal insects that are active in dim light [2,9]; such insects include night-active beetles and moths. It is apparent that at the lowest level of illumination, substantial pooling is usually being undertaken by the visual system of night-active insects both through optical means and through neural/computational interconnections [10].

Spatial pooling ensures that the effective angular resolution of the eye will be lower than that indicated by the density of the optical elements. Given the already limited resolution of insect eyes compared to our own experience, the biological findings indicate that the MW may be a useful cue even with comparatively low-resolution vision systems. Appendix A discusses the extreme limits of low light vision that have been found in insect visual systems, allowing vision well below starlight conditions, as shown in Appendix A Figure A2.

Daytime celestial navigation has been observed and studied extensively in many species. In some insect species such as dragonflies, it has been found anatomically that there is polarisation sensitivity of the dorsal rim area of their compound eyes, suggesting that they are sensitive to the orientation of skylight polarization [12,13]. This characteristic allows them to perceive a clear blue sky as a large polarisation pattern aligned to the angular position of the sun. This perception is useful to a flying insect, since otherwise the sky might provide negligible heading information, either absolute or relative, particularly if the sun is near apex, near the horizon, or obscured by clouds. A comparison between the characteristics of the MW and the sky polarisation pattern can be made. Although the brightness level is quite different, they are both large, low contrast structures that dominate the sky if they are visible.

Moreover, it has been found that the desert ant Cataglyphis [14] and the field cricket Gryllus campestris [15] have the ability to use celestial cues to maintain their heading direction while walking, even when their locomotion is disturbed about the yaw axis.

Compared with daytime movement, it is a challenging task to establish and maintain an inertial or celestial frame heading at night. A moonless clear night sky is much dimmer than full daylight, with a change in light intensity from 0.0001 lux under such conditions to 10,000 lux in daylight [16]. A nocturnal dung beetle in Southern Africa, Scarabaeus satyrus, has been found to orient in very straight lines, transporting its dung balls away from competitors. These night-active dung beetles use the MW as a stellar directional cue when the moon and lunar sky polarisation pattern are absent [2,17,18,19]. The MW is used as a landmark for the duration of an outbound sprint. The aim of the insect is to get away, in a straight line, from other beetles congregating and foraging at the same site, who might attempt to commandeer the painstakingly assembled ball.

It is important to understand the limitations of the approach used by dung beetles orienting at night. It seems that they are using the MW as a short-term heading reference, rather than as a compass that is aligned in some way to the inertial frame. In this regard, the mechanism could also be accurately described as a celestial landmark, although the distinction between a compass and a landmark is not significant given the time frame involved and the behaviour of the beetle.

1.3. Contribution of This Study

There have been past insect biology-inspired studies using celestial cues for navigation. The sky itself can be a reference in day time with sun, clouds and scattering [20], resulting in patterns that are stable [21]. Some implementations have been developed by using sky-polarised light [20] to achieve autonomous navigation both on the ground [22,23] and as part of a flying navigation system on a drone [24]. NASA has even considered use of sky polarisation for navigation in the challenging Mars environment, where the magnetic field is not useful for navigation and where deep terrain features surrounding the vehicle might mask the sun [25].

However, existing technological approaches have not yet considered the MW as a celestial navigation cue to be used in the same manner that S.satyrus dung beetles use it. Due to the size and distinction of the MW, it could be useful for coarse orientation of space craft, robots, aircraft and planetary rovers, even under dynamic or high-vibration situations.

In this paper, we investigate a computer-vision-based navigation algorithm inspired by insect behaviour and physiology that can use the MW as an orientation reference from which it is possible to compute direction signals under low-light conditions. A computer vision approach based on a series of algorithmic operations also allows for the possibility of gaining insight into the challenges and limitations faced by biological systems. Of particular interest when considering the MW is the ecological effect that light pollution and atmospheric pollution might induce.

The MW is a difficult photographic subject under many atmospheric and human geographic conditions, and its position in the sky varies over hours and seasons. Even in regional South Australia, where our laboratory is located, the MW is not always visible on a clear moonless night, depending on humidity and dust. To overcome this shortage of data, our data set consisted of both real photographs of the MW and the outputs from the Stellarium software [26] to generate adequate numbers of images with a variety of conditions, such as location, time and light pollution. This study considered the clear night sky without the weather conditions that would preclude visibility of the MW (clouds, fog or rain).

The intent of this study is to both develop a novel heading sensor and to deepen our understanding of precisely why insects have evolved to use this signal. Although the MW is not visible in many human-modified environments due to anthropogenic light, it is quite visible under clear skies in the low population-density areas of the world. Stars are visible at and above stratospheric altitudes from aircraft and satellites at orbital ranges even during the day. An example environment where it might be most useful is in the sparsely populated polar regions, where magnetic compass orientation is of marginal utility [27]. On Mars for example, the magnetic field is not useful for navigation, leaving a selection of computer vision and celestial approaches, including solar and stellar cues [24,28,29]. The insects that use the Milky Way are important species in their ecosystems, epitomised by the dung beetle. Understanding, simulating and emulating their Milky Way navigation method may help to understand and even quantify the vulnerabilities of the species to anthropogenic light, allowing mitigation measures to be designed.

2. Materials and Methods

2.1. Data Generation

The MW is not visible in most urban centres across the world, either due to light pollution, geographical location, atmospheric conditions or contaminants [30]. Therefore, it is a challenge to capture adequate night sky images with proper exposure [31]. To overcome this limitation, we used a simulator, Stellarium (version 0.22.2) [26], to generate MW test images for processing. Stellarium is an open-source desktop planetarium software that the creators claim is intended for the community of amateur astronomers and for knowledge transfer in transdisciplinary research. This software was created for simulating the celestial sphere based on a given time and location. In other words, this simulator can give the observer a reasonably realistic rendering of the sky with variable sky and viewing settings [26].

There are several key settings in Stellarium that we used in this paper: date, location, MW brightness/saturation, light pollution level (LP), etc. The following image, Figure 4, shows an example of the latitude and longitude settings.

Before we select the light pollution levels in Stellarium, we should consider the contrast available to the nocturnal insect’s visual system. The Bortle scale [32] is a nine-level numerical scale that measures the night sky’s brightness, with 1 meaning an excellent dark sky site and 9 indicating a heavily light polluted site. The corresponding qualitative appearance caused by different levels of light pollution is shown in

Table 1. Light pollution levels.

Level	Title	SQM (mag/arcsec${}^2$)	NELM
1	Excellent dark sky site	21.7–22.01	7.6–8.0
2	Typical truly dark site	21.5–21.7	7.1–7.5
3	Rural sky	21.3–21.5	6.6–7.0
4	Rural/suburban transition	20.4–21.3	6.1–6.5
5	Suburban sky	19.1–20.4	5.6–6.0
6	Bright suburban sky	18.0–19.1	5.1–5.5
7	Suburban/urban transition	18.0–19.1	5.0 at best
8	City sky	<18.0	4.5 at best
9	Inner City sky	<18.0	4.0 at best

, extracted from Stellarium. A sky quality meter (SQM) is used to measure the luminance of the night sky, which is given in the commonly used astronomical units of magnitude per square arc second (mag/arcsec${}^2$).

As we can see from Table 1, different light pollution levels represent different naked-eye limiting magnitude (NELM). For example, level 3 corresponds to the rural night sky where the NELM is 6.6–7.0 and approximate SQM is 21.69–21.89 mag/arcsec${}^2$. There are some sky brightness values with related condition information shown in

Table 2. Night sky brightness values.

Condition	Illuminance (mlux)	Zenith Sky Luminance (mcd/m${}^2$)	Zenith Radiance (mag/arcsec${}^2$)
Overcast natural night	<0.6	<0.2	>21.8
Natural starlit night	0.6–0.9	0.2–0.3	21.4–21.9
Bulge of the MW	N/A	2.71	20.5–21.0
Rural night sky (clear, no moon)	0.7–3	0.25–0.8	20.3–21.6
Rural night sky (overcast)	0.7–9	0.25–0.7	19.0–21.6

from [33]. Some natural environmental light levels can be found; for example, the rural night sky (clear, no moon) zenith sky luminance values are in a range of 0.25–0.8 mcd/m${}^2$, and approximate SQM is 20.3–21.6 mag/arcsec${}^2$, which is in our assumed range for nocturnal insect night vision, as discussed in Section 1.2. Level 4 corresponds to the rural/suburban transition area where the NELM is 6.1–6.5 and approximate SQM is 20.49–21.69 mag/arcsec${}^2$. This study [33] also indicates that the rural night sky (overcast) zenith sky luminance values are in a range of 0.25–2.7 mcd/m${}^2$ and that approximate SQM is 19.0–21.6 mag/arcsec${}^2$, which could also be a suitable level setting for generating our test images. Our rationale is that most of the landscape relevant to humans is somewhat light polluted. Thus, we selected 3 (rural sky) and 4 (rural/suburban transition) as the light pollution level settings [32]. All of the test images use the default MW brightness/saturation setting (brightness: 1, saturation: 1).

Figure 5 provides the azimuthal grid (green line), celestial equatorial coordinate grid (blue line) and an example of the MW image that we used to perform image processing. Figure 6 shows a part of the MW with different settings of the Bortle scale as the light pollution levels in Stellarium.

As discussed above, Stellarium allows for sufficient parameter settings to provide a more flexible way to create night sky images. However, this software cannot create real-time weather environments such as cloud conditions for the simulated sky images; thus, we only considered the clear night sky with different light pollution levels.

2.2. Methodology

It is well established that the brightness variation in the sky caused by the MW is responsible for the orientation behaviour of the S.satyrus dung beetles [18,34], but the actual neural computation involved is not exposed by the behaviour. In this section, we present the development steps of the computer vision method presented in Algorithm 1 that we have developed to extract direction information under low light levels of this large but low-contrast celestial landmark. The objective of the method is to reduce the image to an angle representing the orientation of the MW. The proposed method consists of the distinct steps shown in Figure 7.

Algorithm 1: MW Detection and Orientation

2.2.1. Image Thresholding

The only discernible large object in a clear, moonless night sky is the MW. This suites thresholding techniques that assume that the image contains two pixel groups, those belonging to the foreground and those belonging to the background. Because of the simple computation, robustness and adaptability, Otsu’s technique [35] is extensively used for computer vision applications. The algorithm is used to automatically determine optimal threshold values to segment the background from the foreground area [36,37]. Figure 8 shows a comparison of two additional common thresholding techniques, Ridler and Calvard’s method [38] and Kapur’s entropy method [39]. All three popular image-thresholding approaches achieve similar results in this situation.

The following equations represent Otsu’s thresholding method of an image with L gray levels. Suppose we split the image pixels into two classes $C_0,C_1$ by threshold at level t. The probability of occurrence of level i is given by $\frac{n_i}{N}$

$$p\left(i\right)=\frac{n_i}{N},p\left(i\right)\ge 0,\sum _i^{L-1}p\left(i\right)=1$$

(1)

The probabilities of two class occurrences are given by

$${\omega }_0=P\left(C_0\right)=\sum _{i=0}^t{p}_i=\omega \left(t\right)$$

(2)

$${\omega }_1=P\left(C_1\right)=\sum _{i=t}^{L-1}{p}_i=1-\omega \left(t\right)$$

(3)

The mean of the two classes is calculated based on the following equations

$${\mu }_0=\sum _{i=1}^t\frac{i·p\left(i\right)}{{\omega }_0}=\frac{1}{\omega \left(t\right)}\sum _{i=1}^{t}i·p\left(i\right)$$

(4)

$${\mu }_1=\sum _{i=t+1}^L\frac{i·p\left(i\right)}{{\omega }_1}=\frac{1}{1-\omega \left(t\right)}\sum _{i=t+1}^{L}i·p\left(i\right)$$

(5)

Total mean can be written as

$${\mu }_T=\mu (L-1)=\sum _{i=0}^{L-1}i{p}_i$$

(6)

Otsu is defined as the between-class variance (BCV) that is shown as

$${\sigma }_B^2={\omega }_0{\left({\mu }_0-{\mu }_T\right)}^2+{\omega }_1{\left({\mu }_1-{\mu }_T\right)}^2$$

(7)

The optimal thresholding value of Otsu t is chosen by maximising ${\sigma }_{B^2}$

$${\displaystyle t=max\left\{{\sigma }_{B^2}\right\}}$$

(8)

We used MATLAB R2021a software to calculate the Otsu threshold and to apply the flat morphological structuring element to dilate the object area. The spherical structuring element with a selected radius was used to generate the dilated binary object image. There is an example in Figure 9 that shows the MW detection result of the generated test image shown in Figure 10 with cardinal points.

In Figure 9, the left image is the detection area result that is overlaid on the original RGB test images. The middle image shows the threshold results for the MW area, and on the right is the edge detection result.

2.2.2. Low Redundancy Wavelet Entropy Edge Detection (LRWEEDA)

This method uses a combination of the wavelet transform, Shannon entropy and thresholding to achieve edge detection. LRWEEDA has been developed to be efficient, with low redundancy and noise resilience, and it is well suited to real-time image processing applications [40]. LRWEEDA’s basis in Shannon entropy ensures that its performance in high noise or low signal levels will be consistent and constrained by the information available. There are many edge detecting methods, most notable being the Canny edge detector [41] (which we also used in the results section below), and there are neural-network-based techniques [42]; thus, the selection of LRWEEDA and Canny was a choice from many options. The objective of the algorithm is for it to be used inside the modest computational resources of embedded hardware (such as the Raspberry PI 4 or even the Raspberry PI Zero II); thus, deep-learning-based approaches were avoided despite their high performance [43,44].

For each wavelet decomposition level, vertical and horizontal components, $W_{\psi }^H(j,m,n)$ and $W_{\psi }^V$, are combined and normalised between zero and one to define $\mho (j,m,n)$,

$$\mho (j,m,n)=normalise(W_{\psi }^H(j,m,n)+W_{\psi }^V(j,m,n))$$

(9)

Shannon entropy of an image is defined as:

$$H\left(I(x,y)\right)=-\sum _{i=0}^{L}p\left(I_i\right)logp\left(I_i\right)$$

(10)

where p is the probability of pixel value $I_i$ occurring within $I(x,y)$, and L is the maximum pixel value within $I(x,y)$. To combine Shannon entropy with wavelet decomposition, the Shannon entropy value is calculated at each decomposition level (j). Thus, $I(x,y)=\mho (j,m,n)$ results in

$$H(\mho (j,m,n))=-\sum _{i=0}^{L}p\left({\mho }_i\right) \mathrm{l}\mathrm{o}\mathrm{g} p\left({\mho }_i\right)$$

(11)

The optimal wavelet decomposition level is denoted as $\mho (\beta ,m,n)$. A variable threshold $\mathsf{\Lambda }$ is applied to the image:

$$\begin{array}{l}\forall m,n\in \mho (\beta ,m,n)\\ \mho (\beta ,m,n)=\left\{\begin{array}{ll}\mho (\beta ,m,n)& \mathrm{if}\mho (\beta ,m,n)\ge \mathsf{\Lambda }\\ 0& \mathrm{if}\mho (\beta ,m,n)<\mathsf{\Lambda }\end{array}\right.\end{array}$$

(12)

where $\mathsf{\Lambda }$ is the threshold value and varies in the range of $0\le \mathsf{\Lambda }\le 1$.

2.2.3. Radon Transform

The Radon transform (RT) has been used for extracting or reconstructing angular information in many image processing applications. The Radon transform of an input image $f(x,y)$ can be denoted as $g(s,\theta )$ and is defined as its line integral along a line inclined at angle $\theta$ from the y-axis and at a distance s from the origin shown in Figure 11. The algorithm was fully developed by Toft in [45]:

$$g(s,\theta )={\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }f(x,y)\delta (xcos\theta +ysin\theta -s)\mathrm{d}x\mathrm{d}y$$

(13)

where $-\infty <s<\infty ,0⩽\theta <\pi$. $g(s,\theta )$ indicates a projection that uses the Radon transform of $f(x,y)$ at angle $\theta$.

One useful property of the Radon transform is that the rotation of an image by angle ${\theta }_r$ causes the Radon transform to be shifted by the same amount; thus,

$$f\left(xcos{\theta }_r-ysin{\theta }_r,xsin{\theta }_r+ycos{\theta }_r\right)\leftrightarrow g\left(s,\theta -{\theta }_r\right)$$

(14)

As discussed above, the Radon transform for each $(s,\theta )$ calculates the total intensity of the specific line that pairs the parameters. In other words, the maximum value of the Radon transform corresponds to the location of maximum intensity of the straight line in our detected MW area. The Radon transform was implemented by using the radon function in MATLAB, and the angle $\theta$ was computed at angles from 0° to 180°, in 1° increments. In this study, the implementation of the Radon transform ensures that angle outputs are integer degrees, which limit the resolution of results. Figure 12 shows the location of the peak of the Radon transform and displays a circle over the maximum.

Once the maximum value of $\theta$ has been calculated, we can rotate the original image through the negative of the angle to straighten it, as shown in Figure 13, entirely for comparison purposes, to explore the quality of the result.

3. Results

The algorithm was tested on both simulated images and real night sky images. The Bortle scale levels selected for the simulated images that were used for the processing set were 3 and 4. MW brightness and saturation are set as default (brightness: 1, saturation: 1). The simulated test image results can be seen in Figure 14; the first column provides a brighter and more clear MW shape (brightness: 5), which can be used for comparison between the Stellarium MW area and the detected MW area. The real images we used are from a live sky camera located at Mount Burnett Observatory, Australia [46]. Some additional information is provided about the camera in

Table 3. Information about the All Sky Camera, Mount Burnett, Australia.

All Sky Camera Information
Location	Mount Burnett, Victoria, Australia
Latitude	37.9725 S
Longitude	145.4955 E
Camera	ASI224MC
Exposure	30 s
Computer	Raspberry Pi 3B+

. In order to illustrate the MW shape clearly, Figure 15 shows the MW area with increased brightness (Stellarium MW brightness set as 5).

The results from the live sky camera can be seen in Figure 16. This figure illustrates the MW detection and angle calculation results for real night sky images.

The intent of the algorithm is to simulate the use of the MW as a celestial direction reference. The algorithm was executed on simulation images that resulted from the observer undergoing a continuous rotation rate of 0.5°/s with updates at 10 s intervals, imposing both a rotation and the gradual movement of the celestial hemisphere and MW over time. All the dates and times for this test are set on different moonless nights. Figure 17 shows a small subset of the test images from Stellarium for illustrative purposes: The images were captured from angle $\theta$ to $\theta +315°$, at 45° intervals. All of the images that were used for this angle calculation test were under LP:4 conditions (MW brightness: 1 and saturation: 1). The rotation angle calculation results for three times and locations are plotted in Figure 18.

A Canny edge detector [41] was also tested with comparable results, shown in comparison to the LRWEEDA in both simulated images (Figure 19) and real sky images (Figure 20).

Figure 21 shows the low-resolution images generated from the Stellarium test images (MELO 0227) with the pixel value provided in Figure 22. The original image is 965 × 965 pixels, and the downsampling image resolution is shown in Figure 22. As we can see from Figure 21, in low-resolution scenarios, our proposed method provides reliable navigation cues, even though the resolution was reduced to around 100 × 100 pixels.

Figure 23 and Figure 24 show the MW area that was detected on two sets of test images from Stellarium chosen from the same location but on different dates. In each test set, only the light pollution levels were set from 1 to 7; the remaining settings were the same (location, date, time, MW brightness and saturation). The resolution for all test images for the LP level comparison was 1280 × 920; the figure shows the total pixel values inside the MW area. As we can see from Figure 23, all test images are listed around the graph, and the total pixel values for each LP level were calculated by summing the detected MW area from the binary images.

Figure 25 shows the comparison of two different light pollution levels (LP: 4 and LP: 6) with simulated test image rotation angle calculation results. All test images were generated in Stellarium with the same date, location and MW (brightness: 1 and saturation: 1) settings. For all of the test images, the angle changed 5° for each iteration every 10 s of Stellarium simulated time.

4. Discussion

Overall, the mean accuracy of the method is better than $\pm 2°$ for the sequences tested under LP: 4, shown in

Table 4. Summary of results from the MW compass method and corresponding LP levels.

Sequence	LP	Mean Error	Maximum Error
AAO 01 Jul	4	$\pm 1.71°$	$\pm 4.0°$
AAO 25 May	4	$\pm 0.19°$	$\pm 1.0°$
MELO 27 Feb	4	$\pm 0.58°$	$\pm 1.0°$
MELO 27 Feb	6	$\pm 1.53°$	$\pm 18.0°$

, which is comparable to other azimuthal reference technologies, whether magnetic or celestial. The results degraded under higher light pollution levels. Maximum errors did not exceed $\pm 4°$ under LP: 4 conditions. However, in the presence of light pollution, the angle reference provided by the MW starts to break down; thus, under LP: 6, maximum errors of around $\pm 18°$ emerge. In addition, different features of the MW endure longer under high LP levels, leading to an offset solution compared to the angle measured at higher light levels. Exploring the performance of the method of segmenting the MW (and by analogy, probably any comparable method), it becomes apparent that the higher LP levels render the MW inaccurate, then unreliable, and finally invisible and useless for navigation as the LP level increases.

Unlike the individual points of intensity that represent stars, the MW is a large distributed pattern that is in some ways a complementary detection problem to star detection by celestial navigation systems. There are a number of advantages to this, including robustness against vibration that would blur out individual stars and robustness to loss of focus and other phenomena such as condensation and dust on lenses. At this stage, the system is not robust to partial occlusion by clouds, foliage or buildings, although such obstacles do not exist in every environment for every use case. The system would also not detect when it was not functioning, a very necessary function in a navigation system. These problems might be overcome using a two-stage process of detection and measurement well suited to deep learning.

An interesting outcome of the study was that the problem transferred easily between a simulator that makes some effort to achieve visual realism and real star images. This bodes well for future pattern classification methods, since there will be no limitation caused by availability of data; as long as the likely environmental illuminants and occlusions can be meaningfully incorporated, superimposed or post processed into the simulator output.

Although this study was inspired by the visual behaviour of insects, it still used cameras that are bound by the limitations of conventional electronics. The megapixel resolution of the cameras is significantly more than the mere hundreds to thousands of ommatidia that form the input to nocturnal insect visual systems. A more biomimetic approach would probably make better use of the characteristics of the MW, including its large size, dominant low spatial frequency features and dimness compared to surrounding stars. Orders of magnitude of efficiency and sensitivity may be achieved by a tuned and optimised system.

We did not use a deep learning approach in this initial study because we wanted to understand the characteristics of the problem and of the solution. We also wanted to understand the nature of the data and potential sources of data, since the MW is a demanding photographic subject that would need to be filtered carefully before being inserted into a training data set. Deep learning is likely to be useful for segmenting the MW from other visible objects, such as trees and buildings but also clouds.

From an ecological perspective, we have shown that under low light pollution conditions, insects should certainly be able to see the MW and use this information for direction signals. However, a cautionary observation is that light pollution is the primary risk to this mode of navigation. Nocturnal insects play vital ecological roles; for example, the dung beetle’s role is well documented [47], and moths and night flying insects are variously pollinators and food for organisms higher in the food chain. Anthropogenic light is accepted as a cause of damage to insect populations [48] and yet would also appear to be manageable and largely avoidable. In this work, we have demonstrated how much disruption it can cause to a MW navigation system.

More system integration is required for this sensor modality to be used for true navigation, apart from the vision system being able to detect the presence of reliable MW conditions. An accurate internal clock and almanac for the orientation of the MW in the inertial frame at a location would be required to determine a navigation heading. A reference for the direction of the gravity vector would be required, particularly for space craft and aircraft. However, for short-term heading stabilisation in low light pollution environments, the Milky-Way-based approach presented here should serve a robot just as well as the MW serves nocturnal beetles.

5. Conclusions

Using biology for inspiration and conventional machine vision techniques for execution, we have demonstrated that the MW can be reliably detected and measured for orientation as a suitable input to a navigation system. The method achieved maximum errors of less than $\pm 5°$ and mean errors of less than $\pm 2°$ under the low light pollution test conditions. The study also demonstrated the detrimental effect of stray light on MW-based navigation.

Future research work will consider deep learning and neural biomimetic approaches that may be more robust and less computationally expensive. The method presented is low in computational cost and appears to be reliable; thus, we will soon endeavour to use the method for on board processing of stellar imagery on robotic platforms. Substantial optimisation and robustness can be added to the technique by exploiting the low-resolution aspect of insect vision.