Imagine All the Plants: Evaluation of a Light-Field Camera for On-Site Crop Growth Monitoring

Abstract

The desire to obtain a better understanding of ecosystems and process dynamics in nature accentuates the need for observing these processes in higher temporal and spatial resolutions. Linked to this, the measurement of changes in the external structure and phytomorphology of plants is of particular interest. In the fields of environmental research and agriculture, an inexpensive and field-applicable on-site imaging technique to derive three-dimensional information about plants and vegetation would represent a considerable improvement upon existing monitoring strategies. This is particularly true for the monitoring of plant growth dynamics, due to the often cited lack of morphological information. To this end, an innovative low-cost light-field camera, the Lytro LF (Light-Field), was evaluated in a long-term field experiment. The experiment showed that the camera is suitable for monitoring plant growth dynamics and plant traits while being immune to ambient conditions. This represents a decisive contribution for a variety of monitoring and modeling applications, as well as for the validation of remote sensing data. This strongly confirms and endorses the assumption that the light-field camera presented in this study has the potential to be a light-weight and easy to use measurement tool for on-site environmental monitoring and remote sensing purposes.

1. Introduction

Global warming, a growing world population and a rising demand for food and energy means that new management strategies, not the least of which in the field of precision agriculture and environmental research, are required [1]. As a consequence, special emphasis is placed on the development of more precise environmental monitoring strategies and more reliable climate models [2]. In order to provide recommendations for action and political guidance in a changing world, it is necessary to assess the impacts of climate change and process dynamics in nature with the best possible accuracy and with respect to the inhomogeneity of ecosystems. This especially concerns plant growth and harvest yields in agriculture or adaption strategies for different types of land utilization under changing climate conditions [3]. Until now, numerous studies have been carried out that focus on this particular topic, where the impact of climatic changes on plant growth is predominantly simulated and measured in small laboratory or pilot-scaled experiments, such as in isolated chambers or greenhouse tests. Examples of such extensive surveys are provided by [4,5].

However, a constantly recurring problem with these studies is the small size of the observation plots, low temporal resolution and short investigation periods. This is due to the relatively high investment costs for the field equipment. The prohibitive nature of conducting this research in turn leads to reduced quality and significance of the data. To address this issue, the Global Change Experimental Facility project (GCEF), founded and established by the German Helmholtz Centre for Environmental Research, follows a larger scale approach, which, despite sounding rather mundane in theory, is actually extremely challenging to implement in practice, since the applied monitoring strategies have to be field-applicable and fully comprehensive. Therefore, new imaging methods and monitoring strategies have to be investigated that combine the dual advantages of low financial costs and lower methodological difficulty, which could also potentially be particularly appropriate for research areas related to agriculture, remote sensing and the modeling of plants, plant traits and ecosystems by providing on-site references or in situ ground truth data for remote sensing applications.

In particular, the extraction of morphological traits and the comprehensive monitoring of plant growth and growth dynamics based on imaging methods have become important working areas in the field of environmental research and ecosystem observation in forest and agricultural sciences [6,7,8,9]. Another important aspect concerns the need for in situ data in ground truth sampling to evaluate and calibrate remote sensing data [10].

In previous studies, most of the presented imaging techniques used to derive the morphological traits of plants were comparatively cost-intensive and focused on either stereo vision [7,9,11,12,13,14] or were laser scanner-based [15,16,17,18,19] approaches, which required a high level of methodological effort during field measurements and, therefore, resulted in reduced field capability [20,21].

In order to fulfill the requirements of an appropriate on-site monitoring tool, an ideal system should be able to derive the aforesaid plant characteristics in an automated, inexpensive and easy to use way regardless of the ambient conditions. This includes an appropriate calibration, as well as a stable performance to ensure reliable and repeatable results even under harsh conditions. To overcome these challenges, we tested an innovative and inexpensive light-field camera, the Lytro LF (Light-Field) (see Figure 1), in a large-scale field experiment, designed to illustrate the benefits and drawbacks of this technique via a feasibility and “proof of concept” study. For this purpose, it is of particular interest whether or not the light-field technique is applicable and can provide in situ three-dimensional information about the phytomorphology and growth dynamics of the plants, e.g., plant heights or plant canopy parameters over time.

Table 1. Lytro specifications [22].

Camera Unit	Specification	Property
Lens	Focal Length	43–344mm
	Zoom	8× optical
	Aperture	Constant f/2.0
Image Sensor	Sensor Type	CMOS
	Light-Field Resolution	11 Megaray (the number of light rays captured by the light-field sensor)
	Active Area	4.6 mm × 4.6 mm
Image	Format	Light-Field Picture (.lfp)
	Aspect Ratio	01:01
	2D Export Resolution	1080 × 1080 pixels (approx 1MP peak output)
	File/Picture Storage	350 files 8 GB
Exposure	Modes	Full Auto, Full Manual, Shutter Priority or ISO Priority
	Shutter Priority	$\frac{1}{250}$–8 s
	ISO Priority	80–3200
	Exposure Lock	Yes
	Neutral Density (ND) Filter	4-stop
	Control Interface	Tap on touchscreen
Screen	Touchscreen	Yes
	Size	1.52 ${}^{\prime \prime }$ (diagonal)
	Screen Type	Back-lit LCD
	Live View	Yes
Playback	In Camera Picture Review	Yes
Power	Battery	Built-in, rechargeable long-life lithium-ion
	Battery Charging	Via Micro-USB to computer or Lytro Fast Charger
External	Controls	Power button, Shutter button, Zoom Slider, Touchscreen
	USB	Micro-USB
	Tripod Socket	Available via Lytro custom accessory mount, sold separately
Miscellaneous	Software	Includes LYTRO Desktop for importing, organizing, processing and interacting with living pictures. See more in the LYTRO Desktop Fact Sheet
	Wireless	802.11 b/g/n, Wi-Fi Protected Access (WPA2)
	E-Waste	RoHS certified (Restriction of Hazardous Substances Directive 2002/95/EC)
Materials	Lightweight anodized aluminum with silicone grip Camera Kit Includes: LYTRO Magnetic Lens Cap, lens cleaning cloth, wrist strap, USB cable for data transfer and charging and Quick Start guide
Dimensions	41 mm × 41 mm × 112 mm
Weight	214  g

contains detailed information about the light-field camera Lytro LF used in this study. In the next section, the description of the light-field optics and the working principle of the camera itself are examined once again in an overview.

2. Preliminary Consideration and Definitions

Since the optics and mechanisms of image capturing are complex, the following simplifications are proposed. The object space will be simplified to a two-dimensional scene where light is defined as a scalar value, spreading in a straight line [23]. In this case, light from a certain point of a scene can be described as shown in Figure 2. When initially assuming that a camera can be regarded as a pinhole camera, any object placed in front of it would create a real image located in the plane of convergence [24]. This simple set up is shown in Figure 2a. It must be pointed out that this real image projection provides no useful depth information about the scene [25] unless a second pinhole camera is added, as is done in Figure 2b.

Based on these two images (Figure 2b), it is possible to extract further information about the structure and the depth of the scene. This stereoscopic effect is used in binocular stereo systems or stereo vision systems. Instead of using two or more single cameras to increase the amount of view points, it is also possible to move the camera (see Figure 2c), a technique also known as structure from motion or motion parallax [24].

If now a lens is placed at the pinhole, the light that hits the plane is captured from an endless variety of view points located at the lens plane (see Figure 2d). In other words, different view points refer to the different images being visible through different positions at the lens aperture plane. Usually, the light is captured as an averaged signal at the sensor plane without respect to its direction or origin [24]. This means, the incoming light from each point of the aperture plane contains more optical information than the averaged signal projected on the image plane represents. The entirety of this optical functions (intensity, direction, wavelength) is summarized and called plenoptic functions [24,26]. To retrieve this optical information, a simple assumption can be made. Figure 3a shows an object in the focus plane of the camera creating a sharp image of the object. If the object is for instance located closer to the camera, the resulting image would be out of focus, and the object would appear wider on the image plane (see Figure 3b).

The same phenomena would appear if the object were placed farther from the focal plane. To distinguish whether the object is close or far away from the camera (or rather the focal plane), it is assumed that the camera has a non-centric aperture, as is drawn in Figure 3c. Then, the object would only appear on one side of the image plane, due to the rays of light passing through the lens. Depending on the selected aperture, as well as on the distance from the camera, the object will appear either on the left or the right side of the optical axis, while the displacement between the object projection and the optical axis is dependent on the selected aperture [24].

By knowing this, it is possible to derive the depth of the scene from the known displacement of the aperture at the lens plane and the displacement of the object from the optical axis at the image plane. Figure 4 contains a simplified geometry of the relation between the displacement and the object distance described above.

The estimation of the object distance d (here simplified for a single object point) or rather the depth of the scene can be calculated by the known displacement of the aperture v and the aperture of the image h. Therefore, the triangle between the focal point beyond the sensor plane and the image displacement h can be assumed as being similar to the triangle of the focal point and the aperture displacement v, which leads to Equation (1) [24].

$$\frac{v-h}{l}=\frac{v}{g}$$

(1)

This can be transformed to:

$$\frac{1}{g}=\frac{1}{l}\left(1-\frac{h}{v}\right)$$

(2)

and in combination with lens equation:

$$\frac{1}{f}=\frac{1}{g}+\frac{1}{d}$$

(3)

the following expression can be formed:

$$\frac{1}{d}=\frac{1}{f}-\frac{1}{l}\left(1-\frac{h}{v}\right)$$

(4)

Equation (4) can be used to determine the distance d by the known camera parameters and the measured displacement h in the image [24].

In theory, this could be achieved by separating the area of the sensor plane into an array of sub-systems, for instance, small pinhole cameras. Now, each pinhole camera captures light from a certain direction or rather from a certain position in the aperture plane of the lens. Since a pinhole camera is always capturing the incoming light in the form of a real image (see Figure 2a), the light rays that are now hitting the sensor plane can be described according to their direction and intensity [24].

Figure 5 shows the basic design of such a simplified plenoptic camera and the principle of capturing the light field for three different cases, assuming that each camera has a resolution of three pixels (labeled as $u,v$ and w).

Particular attention should be given to the opening of each pinhole camera towards the center of the main lens. The position of the pinholes slightly changes due to the fact that each pinhole camera is sensitive for a specific subset of the incoming light through the main lens. Therefore, each camera is aimed at the center of the lens plane (see the blue lines in Figure 5). In the first case, the object point is located at the focal plane of the main lens. As a consequence, the resulting image is projected inside the center pinhole camera, which is solely responsible for the corresponding sensor signal. If the object is placed either further away or closer than the focus plane of the main lens, the corresponding signal would be caused by three pinhole cameras (for this simple design). However, according to the angle of incidence, only specific pixels are responding, which leads to a measurable displacement [24]. Finally, this displacement can be used to determine the distance of the object according to the above-mentioned Equation (4). In practice, the technical realization of the above described consideration is done by using a micro lens array instead of the assumed pinhole cameras.

3. Materials and Methods

3.1. Hardware Setup

In late 2011, the U.S. company Lytro Inc. launched the first consumer light-field camera, the Lytro LF (Lytro LF 8 GB, Lytro Inc., Mountain View, CA, USA). The release of this camera gained much publicity and attention, as it represented a revolutionary change in digital photography. The most fascinating aspect is the possibility of refocusing after a picture has been taken. The figures below illustrate the refocusing effect, from a very close focal point to a far focal point within the same image file (see Figure 6).

To be more precise, the fact that the focal point can be changed after capturing an image indicates that the camera obtains spatial information about the object as it captures the entire light-field of an image (direction and intensity of the incoming light), which in practice allows for a variety of possible applications, e.g., depth estimation or three-dimensional visualization. The depth information can be extracted from a depth map (see Figure 6d), which is generated by the Lytro software. Due to its low price (the device cost only $99 in 2015) and small dimensions, the Lytro LF is an excellent piece of technology, which allows us to explore the behavior and possible uses of a low-cost light-field camera, e.g., for environmental monitoring purposes.

Although the basic idea of light-field vision is more than 100 years old [27,28], only a few light-field cameras are available on the market [22,29,30]. A literature review on light-field technology clarifies that the majority of publications deal with the theoretical background of light-field cameras [24,26,31,32]. One of the few published studies to date that deals with the applications of light-field cameras focuses mainly on clinical photography and documentation of the growth stage of skin cancer [33]. In another study, the Lytro LF was used to diagnose and identify abnormalities in pediatric eyes [34]. However, due to the novelty of light-field photography, the cameras are not as widely used, which could provide an explanation for the limited number of studies carried out on this topic.

Nevertheless, the Lytro LF seems suitable for environmental monitoring especially with regard to ease of use and the small size of the camera. For this reason, the Lytro LF was used in this study, both for conceptual investigation and to obtain a practical proof that the device can be used for the monitoring of crop growth under field conditions.

3.2. Camera Preparation and Calibration

In order to carry out the proposed experiments, additional hardware was required. As the Lytro LF was designed as a consumer camera intended for everyday use, it does not support any external triggering by default. Therefore, some modifications were necessary to make the camera remotely controllable. To this end, the camera was disassembled, and two cable stands were soldered onto the $\raisebox{1ex}{$\mathrm{On}$}\!\left/ \!\raisebox{-1ex}{$\mathrm{Off}$}\right.$ button, as well as the shutter. In addition, a switching device was designed based on two opto-couplers. In combination with an $\raisebox{1ex}{$\mathrm{A}$}\!\left/ \!\raisebox{-1ex}{$\mathrm{D}$}\right.$ converter (NI USB-6008, National Instruments, Austin, TX, USA), this switching device allows the software-controlled operation of the camera. The setting of the output channels (a0, a1) of the $\raisebox{1ex}{$\mathrm{A}$}\!\left/ \!\raisebox{-1ex}{$\mathrm{D}$}\right.$ converter to either “logic high” or “logic low” via the control software (LabView, Version 2009, National Instruments, Austin, TX, USA) results in switching on or off the camera, as well as capturing an image.

The most significant aspect when preparing the field experiment was the calibration of the depth estimation or rather the evaluation of the depth map quality. The depth map calibration was done empirically by placing the Lytro LF perpendicular and with a distance of $d=1.00\mathrm{m}$ towards a calibration plane. Prior to the experiment, the Lytro LF was activated, and the focal point of interest was set at the calibration plane. Between the plane and the Lytro LF, a movable object was inserted, starting at the camera housing ($d=0.00\mathrm{m}$). In this context, it should be mentioned that the object, as well as the background (here, the calibration plain) should be visible. The depth estimation works very sufficiently whether the object is placed in the center of the image or at the rear as long as the field of view contains enough visible structural geometry. Figure 7c clearly shows that the whole object was assigned properly over its entire area. However, very smooth and homogenous objects cannot be recognized adequately by the Lytro LF. Once the picture was taken, the object was moved further away with a step width of $\mathrm{\Delta }d=0.01\mathrm{m}$ until the calibration plane was reached (see Figure 7).

Subsequently, the generated data were evaluated using MATLAB (R2014b, The MathWorks, Inc., Natick, MA, USA). As a result of the experiment, a relation between the grey values in the depth map and the object distance was found (see Figure 8), which remains unaffected even when the camera is turned off and on again.

Figure 8 illustrates the corresponding grey value in the depth map compared to the object distance in the range of $d=0\mathrm{cm}$ to $100\mathrm{cm}$. It is clearly stated that for larger distances, the same grey values are allocated for different distances indicating the limited range of proper distance recognition. Of particular note is that object distances of less than 10 cm are assigned to the same grey value, as well. The exponential relation given in Figure 8 allows calculation of the object distance with a good degree of accuracy ($R^2=0.9343$). Figure 8 also shows that the measurement range for obtaining good results is limited to close range applications, e.g., 10 cm to 50 cm. Figure 9 shows the depth estimation error depending on the object distance. This further underscores the very appropriate performance of the Lytro LF within the range, as mentioned above.

3.3. Experimental Design

The field experiment took place at a project site of the Helmholtz Centre for Environmental Research in Bad Lauchstädt from 7 August 2014 to 7 September 2014 (see Section 3.4). As a first step, the light-field camera had to be prepared for the field experiment. For reasons of simplification, a custom housing was designed for the Lytro LF. Thus, a waste water pipe was prepared and had, due to its excellent optical and mechanical properties, a fused quartz window added to it. To keep the Lytro LF in position inside the pipe, two retaining brackets were inserted, which can be screwed from the outside. Once the Lytro LF was placed inside, the switching device was mounted, and the housing was closed with a screw cap. Here, a cable gland was used for tension relief and to ensure the housing remained watertight. In order to avoid a build-up of moisture and humidity in the interior, several bags of silica gel were placed inside the housing of the camera. Considering the limited depth resolution of the Lytro LF (established during calibration), the camera was mounted on a tripod at one meter height within the plot (see Figure 10). The cable was fixed, to avoid cable movement, and fitted to a distribution box, where a computer and the $\raisebox{1ex}{$\mathrm{A}$}\!\left/ \!\raisebox{-1ex}{$\mathrm{D}$}\right.$ converter for controlling the Lytro LF were stored.

Since the Lytro LF was fully calibrated in the laboratory, use of the camera in the field turned out to be very easy. Besides the installation of the stand and adjustment of the camera itself, only the cable between the switching device and the controlling unit had to be connected. From this point on, the system was running automatically and very stably during the whole measurement campaign. Here, it should be noted that the camera was not driven by any external power supply. After two months in the field and more than 200 images taken four times a day, the battery charge still remained at over 40 %. This was achieved by using an optimized switching procedure, which ensured that the camera was not turned on for more than five seconds per image acquisition.

Figure 11a shows the field of view of the Lytro LF related to the corn rows. Expressed in real-world dimensions, the Lytro LF covered an area of $A\approx 0.17{\mathrm{m}}^2$. In order to evaluate the feasibility and quality of the plant height estimation, the plant height of the crop was also measured manually. Determination of the plant height was achieved by measuring the distance between the soil surface and the highest point of the arch of the uppermost leaf within the field of view of the Lytro LF [35]. The tips of the next emerging leafs above were not measured since the steeply rising tips are weakly captured during the image recognition.

Processing of the data was performed in a mainly automated way. Since the Lytro LF stores the light-field images in its own format, importing the images from the camera was performed using the Lytro software. During this transfer, the devices began to process the images (depth map calculation), which turned out to be time consuming given the size of the dataset. Once the images were computed, they could be exported and further processed, which was performed by MATLAB and by using the known correlation between the grey values in the depth map and the object distance from the preliminary calibration (see the program

Algorithm A1. Source code plant height measurement Lytro.

in Appendix A). This post-processing is fully automated and does not require any input parameters (which was intended from the beginning of this investigation). The results of the algorithm programmed in MATLAB show the plant coverage and the plant heights in order, which depicts the dynamics of plant growth over time.

3.4. Experimental Sites Description

The experiment was performed at the test site of the Global Change Experimental Facility project (GCEF) of the Helmholtz Centre for Environmental Research [6], a large-scale outdoor test facility located in Bad Lauchstädt (Saxony-Anhalt, Germany). The project site of the GCEF consists of 50 plots, which are covered by a 5m high metal construction that covers a surface area of about $384{\mathrm{m}}^2$ per plot. The plots are distributed over ten blocks, five of which are equipped with temporarily-closing mobile roofs and side panels, which allows the simulation of expected regional climate conditions and to assess the effects and consequences of climate change on different forms of land utilization [6]. In order to measure changes and any possible amendments to plant growth or the biocenosis in general, an applicable imaging technique is needed, especially to help improve the assessment of different impacts and the influences of climate change on the growth dynamics of plants. The field experiment of this study was installed on a pilot-scaled maize field (Zea maize), which covers a surface area of about $384{\mathrm{m}}^2$. Since the maize was specially sown for this study in late July 2014, the maximum plant height was about 1 m at the end of the measurement campaign. At this growth stage, the flag leaf does not yet exist, which is why in this study, normally, the top leaf represents the tallest part of the plant. This is important when comparing manually-measured plant heights and the estimations derived by the light-field camera.

4. Results

4.1. Canopy Measurement

The canopy measurement was carried out by developing a green filtering algorithm within MATLAB for automated leaf detection (see the program Algorithm A1 (lines 106–121) in Appendix A). In order to illustrate the quality of the green filtering, an example in the form of two raw images and the filtered results is given in Figure 12. Although canopy measurements based on RGB cameras are common, here, it should be noted that the images acquired by the Lytro LF are processed and computed using internal algorithms. Laboratory tests with a calibration grid have pointed out that images acquired by the Lytro LF show negligible lens distortion and can be compared to rectified images, which is particularly advantageous for photogrammetry or machine vision purposes. Moreover, this could be used to determine more specific plant characteristics, such as leaf area index or biomass.

Based on the green filtered images, it was possible to estimate the fractional vegetation cover. The result of this estimation is given in Figure 13. Here, each point represents the ratio between pixels assigned to be vegetation and non-vegetation during the field campaign. This illustrates that the coverage ratio is increasing constantly during the measurement campaign except for a few outliers. At the end of the observation, the canopy values declined, which is due to the fact that the plants have reached the camera housing. Therefore, the camera was very close to the top leaves while measuring the lower leaves and surrounded by a higher amount of soil, which caused slight damage to the vegetation cover curve.

4.2. Plant Height Measurement

Besides canopy coverage, the Lytro LF was mainly used to measure the plant heights of the maize. During the field campaign, four images were taken daily (9 a.m., 12 p.m., 3 p.m., 6 p.m.). The following table contains the estimated plant heights for three days on which the plant height was also measured manually (see

Table 2. Manually-measured plant heights in comparison with the automated estimation of the Lytro LF.

Date	Plant Height in cm
	Manually-Measured	Lytro LF Estimation
20 August 2014	42.5 ± 1.4	54.7 ± 7
28 August 2014	66.9 ± 4.1	70.6 ± 5
4 September 2014	93.4 ± 5.3	86.7 ± 1
10 September 2014	127.7 ± 14.7	-

). Since five maize plants were within the field of view of the Lytro LF (see Figure 11a), the value of the manually-measured plant height shown in Table 2 consists of the average plant height of all plants accompanied by the standard deviation. The Lytro LF value consists of the average value of the $0.01\%$ quantile accompanied by the inaccuracy of the measurement range known from the calibration (see Section 3.2).

The data obtained during the three investigation days given in Table 2 correspond to the data represented in Figure 14, beginning with 20 August 2014. Here, an RGB image, as well as the corresponding depth map is given for each day. As Figure 14 illustrates, the depth maps provide a good approximation of phenology and plant height during the measurement campaign. It is evident that the maximum values are increasing from Figure 14b (plant height approximately 50 cm) to Figure 14f (plant height approximately 90 cm). The figures also indicate that there are some incoherent areas within the leaves of the plants. This is discernible by the dark blue spots inside the leaf shape (e.g., Figure 14d) and due to the fact that these areas were overexposed leading to a reduced structure of the object within the image. Furthermore, the figures demonstrate that shadows (which are visible in the RGB images) do not affect the quality of the depth maps, which thus represents a fundamental finding of the experiment and proves the feasibility of not using artificial illumination in the field. Besides the depth map, the RGB images give an idea about the growth stage and situation within the plot. This could be valuable additional information, especially with regards to devising a comprehensive and wide-ranging monitoring strategy.

Figure 15 gives an impression of plant growth during the entire measurement campaign, gathered by the Lytro LF. Here, again, only the maximum values per observation were recognized (tallest plant part). In order to evaluate the data more thoroughly, a quantile plot of the plant growth is given. The plot confirms the assumption, which was mentioned above, that the depth estimation is only reliable for plant heights in the range of $50\mathrm{cm}<$ plant height $<90\mathrm{cm}$. For values out of this range, the horizontal path indicates another distribution, which is likely due to an estimation error. The course of the plant height estimation shows a high dispersion of the values, especially for plant heights below 50 cm. It is conspicuous that during the whole campaign, no values below 45 cm were measured, which is again related to the fact that objects far away from the camera cannot be detected properly. Here, again, it should be mentioned that the image recognition is weaker for object distances larger than 45 cm leading to an estimation error of up to 13 cm (see Figure 9). To this end, the horizontal paths within the data are a result of decreasing depth resolution and more improper value assignment of ±10 cm. Another difficulty can be found in the measured object. During the early growth stages of maize plants, top leaves grow straight and vertically upwards before they curve downwards when a new plant stem appears. This succession of sprouting and bending of the leaves can cause an unsteady growth rate. Moreover, it should be emphasized that the experiment has taken place under field conditions where the plants were influenced by wind, dryness and humidity. Moreover, the data derived by the Lytro LF can be used to depict growth dynamics in the form of a temporal histogram plot. As the colorbar exemplifies, each recording is transformed to a histogram, and the intensity of the color indicates the relative share of the population. The step width is 10 cm in a range from 0 cm to 100 cm. Especially for the second and the third observation point, the curves show a similar course, while the plant height was overestimated at the first point. This is due to the fact that the plants where in the moderate measurement range of the Lytro LF of more than 50 cm. Outside of this range, the underlying depth map shows grey values, which can correspond to several depths as mentioned in the calibration section (see Section 3.2 and Figure 9).

In addition, Figure 15 also illustrates manually-measured plant heights in comparison to the estimations by the Lytro LF. The blue curve represents the manually-measured plant height of the maize according to Table 2. Unfortunately, an unexpected fast plant growth at the end of the experiment was responsible for the low amount of comparable measurements since the plants exceeded the measurement distance of 100 cm, leading to incorrect values and depth estimations. However, the temporal plot allows a significant insight into the growth dynamics of the crop, since all visible plant parts are taken into account. This allows the discussion of the plant growth dynamics depending on different plant levels or leaf stages.

4.3. Time Lapse Plant Observation

Based on the acquired depth maps, it is possible to illustrate the growth dynamics of the plants in the form of a video or time lapse animation. The animation demonstrates an intelligible and easily accessible way to investigate differences and anomalies of the leaf arrangement, phenotypes or the plant growth in general as a function of time (see Figure 16). The short video points out the advantages of an automated long-term on-site monitoring since it clarifies the varying conditions and the phenotypical changes of the plants during the field experiment. Therefore, the author would like to highlight the time lapse animation video, which can be found in the multimedia section.

5. Discussion

Based on the evaluation of the laboratory experiment, the most suitable distance for a good depth estimation is between 10 cm and 50 cm away from the Lytro LF, which is a constraint of the small sensor size leading to a very small parallax. Therefore, the stereoscopic basis of the device is limited (sensor size 6.451 mm × 4.603 mm) and very small compared to a common stereo vision system. For this reason, the estimation of plant heights and, moreover, the generation of a digital plant model were expected to be inaccurate for distances of more than 50 cm away from the camera. Due to the limited range of the Lytro LF, the camera was installed at only a 1-m height. As a result, the plants reached the Lytro LF housing on 3 September 2014. This also clearly indicates the limitations of the Lytro LF.

To improve this, a larger sensor would be useful. This has already been implemented in the successor of the Lytro LF, the Lytro Illum. However, the Lytro Illum was not available at the time the experiment was conducted. The prices for such devices will fall and, so, the limits of possibility. Another way to improve the depth resolution is to increase the pixel size and the amount of pixels under each micro lens, or more precisely, increase the disparity, which is responsible for the depth resolution and light-field capability of the camera in general. Since production of the micro lens arrays in combination with a high resolution sensor (amount of pixels) is expensive and relatively complex, this would increase the costs for such a light-field camera drastically.

Considering the low price, it may be sufficient for the moment to simply change the position of the Lytro LF gradually depending on plant growth or to install several cameras at different heights in the field. This simple modification would ensure that the object of interest or the top leaves are always within an appropriate measuring range of the camera (or cameras).

However, as a result of this study, the relative canopy coverage and plant height were successfully estimated based on the light-field images acquired by the Lytro LF, which allowed us to gain initial insights into the plant population of the plot. Furthermore, the Lytro LF is an appropriate test object for investigating the feasibility of a light-field camera as a monitoring tool under field conditions. To this end, the results illustrate that this device is a very promising tool that can be used for the monitoring of plant growth, plant traits and especially the identification of process dynamics. This, above all, becomes apparent through the comparison of manually-performed measurements and the estimations generated by the Lytro LF. Even though the first value was overestimated by more than 17 cm, the other results were in a good accordance. As a consequence, this confirms the assumption that this light-field camera has the potential to be a suitable measurement tool for environmental monitoring purposes and is a promising alternative compared to cost-intensive imaging techniques.

Although the overall course of plant growth exhibited a fairly wide dispersion, in consideration of the simple experimental design, the results achieved have to be regarded as a helpful insight into the growth dynamics (see Figure 15). In addition, this indicates that initial estimations concerning plant height monitoring can be derived and verified by a light-field camera, even by an inexpensive consumer product like the Lytro LF.

6. Conclusions

The most significant specification of the Lytro LF is the ability to provide three-dimensional information about the image. To this end, a laboratory experiment was performed to learn about the capabilities of the Lytro LF and the depth resolution. The test demonstrated a relation between the object distance and the grey values in the corresponding depth maps of the light-field images generated by the Lytro software showing a very appropriate performance for short range applications up to 50 cm. In this manner, the Lytro LF can be utilized to extract real-world distances. Moreover, the field test results showed that the Lytro LF is suitable for the monitoring of crop growth under field conditions; however, with certain restrictions. Thus, it seems that the Lytro LF has the potential to become a promising alternative to conventional stereo vision solutions. Furthermore, the Lytro LF can be regarded as a stand alone technique, since no further enhancements are needed to fulfill the depth estimation requirements, even under field conditions.

In conclusion, the Lytro LF approach can be used as an inexpensive measurement tool for environmental monitoring purposes, although its scope is limited to close-range applications. The need for higher pixel resolution and image depth may be accomplished by newly-developed upcoming light-field cameras, which introduce a variety of possible applications. Furthermore, it is expected that the costs for such cameras will decrease over time. As a consequence, new applications will be discovered in the field of remote sensing, environmental monitoring and forest and agricultural sciences in general.

Furthermore, special attention should be given to the simplification of the experimental design and the data processing, especially the opportunity to provide three-dimensional information without any additional requirements during field measurements based on one single shot, which represents a decisive and cost-effective contribution in the area of environmental research and the monitoring of plant growth.