Evaluating the Precision and Accuracy of Proximal Soil vis–NIR Sensors for Estimating Soil Organic Matter and Texture

Abstract

Measuring soil texture and soil organic matter (SOM) is essential given the way they affect the availability of crop nutrients and water during the growing season. Among the different proximal soil sensing (PSS) technologies, diffuse reflectance spectroscopy (DRS) has been deployed to conduct rapid soil measurements in situ. This technique is indirect and, therefore, requires site- and data-specific calibration. The quality of soil spectra is affected by the level of soil preparation and can be accessed through the repeatability (precision) and predictability (accuracy) of unbiased measurements and their combinations. The aim of this research was twofold: First, to develop a novel method to improve data processing, focusing on the reproducibility of individual soil reflectance spectral elements of the visible and near-infrared (vis–NIR) kind, obtained using a commercial portable soil profiling tool, and their direct link with a selected set of soil attributes. Second, to assess both the precision and accuracy of the vis–NIR hyperspectral soil reflectance measurements and their derivatives, while predicting the percentages of sand, clay and SOM content, in situ as well as in laboratory conditions. Nineteen locations in three agricultural fields were identified to represent an extensive range of soils, varying from sand to clay loam. All measurements were repeated three times and a ratio spread over error (RSE) was used as the main indicator of the ability of each spectral parameter to distinguish among field locations with different soil attributes. Both simple linear regression (SLR) and partial least squares regression (PLSR) models were used to define the predictability of % SOM, % sand, and % clay. The results indicated that when using a SLR, the standard error of prediction (SEP) for sand was about 10–12%, with no significant difference between in situ and ex situ measurements. The percentage of clay, on the other hand, had 3–4% SEP and 1–2% measurement precision (MP), indicating both the reproducibility of the spectra and the ability of a SLR to accurately predict clay. The SEP for SOM was only a quarter lower than the standard deviation of laboratory measurements, indicating that SLR is not an appropriate model for this soil property for the given set of soils. In addition, the MPs of around 2–4% indicated relatively strong spectra reproducibility, which indicated the need for more expanded models. This was apparent since the SEP of PLSR was always 2–3 times smaller than that of SLR. However, the relatively small number of test locations limited the ability to develop widely applicable calibration models. The most important finding in this study is that the majority of vis–NIR spectral measurements were sufficiently reproducible to be considered for distinguishing among diverse soil samples, while certain parts of the spectra indicate the capability to achieve this at α = 0.05. Therefore, the innovative methodology of evaluating both the precision and accuracy of DRS measurements will help future developers evaluate the robustness and applicability of any PSS instrument.

1. Introduction

Soil texture and soil organic matter (SOM) affect plant nutrient availability. Measuring these properties using conventional laboratory methods is laborious and time consuming [1,2]. Wherever possible, the use of proximal soil sensing (PSS) is expected to replace conventional laboratory methods to help reduce time- and labor-intense operations during soil sampling and analysis [3,4]. Among different PSS technologies, diffuse reflectance spectroscopy (DRS) has been deployed to conduct rapid soil measurements in laboratories and in situ. DRS is an indirect analytical technique [5] that was proposed as an alternative to laboratory tests that would allow for relatively inexpensive and rapid measurements of soil properties.

A soil spectroscopic system is based on the sample’s absorption of electromagnetic radiation at wavelengths in the ranges of 100–400 nm (UV), 400–700 nm (vis), 700–2500 nm (NIR and SWIR regions), and 2500–25,000 nm (mid-IR) [6]. Soriano-Disla et al. [7] reviewed a number of studies that simultaneously determined different soil attributes using this method. For example, the authors of [8] simultaneously determined soil organic carbon (SOC), moisture, and total nitrogen (TN) using NIR spectroscopy. The authors of [9] simultaneously determined pH, clay, silt, sand, SOC content and cation exchange capacity using mid-IR.

Similarly, numerous studies support the use of diffuse reflectance spectroscopy to simultaneously assess various soil properties in the laboratory as well as directly in the field [10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34]. The Veris^® P4000 (Veris Technologies, Inc., Salina, KS, USA) is an example of a ruggedized hyperspectral instrument developed to obtain vis–NIR spectra at multiple depths within the soil profile. The data obtained from similar instrument was used previously to predict soil phosphorus, soil texture and SOM content [35,36]. In addition, the authors of [37] showed the potential use of similar probing instruments to produce three-dimensional digital soil maps of agricultural fields, by integrating multiple proximal sensor data obtained from different sensing methods.

Since the use of spectroscopic technique to predict the quantities of soil properties is indirect, it is important to develop calibration models using data-specific pedo-transformation models against measurements obtained according to standardized laboratory protocols [3]. Depending on the linear and nonlinear data types, a variety of simple to complex calibration models were studied by various researchers through the performance of multivariate data analysis, e.g., principal component regression (PCR) [16], partial least squares regression (PLSR) [38], artificial neural networks (ANN) [39,40], and support vector machines (SVM) [41], to name a few.

However, a good calibration model can be developed only when a good set of data is obtained from the vis–NIR instrument. This is especially the case when soil is made up of heterogeneous material; the quality of soil spectra can be affected by the level of soil preparation. Generally spectral measurements need to be accurate and precise representations of the targeted material (soil in this case). However, there are a variety of factors that affect the quality of spectral measurements obtained using vis–NIR instruments in both the laboratory and in the field; for example, viewing geometry, orientation of measurement, spectral averaging and calibration of the spectral data [42,43,44]. In addition to this environment, excess water content in soil, and excess soil aggregates and debris can also influence the data collection procedure [38]. The sampling process is also known to influence measurement uncertainty [45].

Therefore, it is clear that the data acquisition and processing may influence the performance of the calibration models [46,47,48,49].

To help improve data acquisition procedures in DRS, it is important to understand the random fluctuation of soil spectra and their derivatives from one scan to the next. As with any measurement, the quality of soil spectra obtained using DRS while predicting any soil attribute can be evaluated using precision and accuracy. Precision (reproducibility) and accuracy (predictability) are considered to be very important characteristics of a given measurement system [50,51]. Where accuracy is not a quantity and precision has a much more complex meaning, both can be described as the difference of the instrument reading to the standard, and the degree of random variations in the instrument’s output while measuring a constant or similar objects (the same soil in the case of the research discussed in this paper) under specified conditions [52]. Other way around inaccuracy may also be defined as combining both bias and imprecision. Where bias is another quantitative term describing the difference between the average of measurements made on the same object (same soil in the case of this research) and its true value [53].

As is illustrated in Figure 1, repeated measurements of soil spectra for a given soil sample can significantly vary from one measurement to another. These differences can be associated with (1) random noise within the data acquisition process that includes inconsistencies of the soil detector interface and (2) soil micro-variability that can occur at very small separation distances. In practice, multiple measurements and the averaging of several consecutive soil spectra were used to diminish the effect of random noise [54]. However, it should be noted that certain parts of the soil spectra, as well as particular parameters constructed using a combination of spectral data values, can be relatively repeatable from spectrum to spectrum. It was also observed that some of these soil spectra parameters do not vary substantially when testing the same soil, but vary considerably when using soil samples with different attributes; therefore, they might be the best candidates to be used for the site-specific calibration process.

The main objective of this research was twofold: First, to develop a novel method to help improve data processing, focusing on the reproducibility of individual soil reflectance measurements and the transformations of the visible and near-infrared (vis–NIR) kind, obtained using a commercial portable soil profiling tool, and to determine their direct link with a selected set of soil attributes. Second, to assess both the precision and accuracy of the vis–NIR hyperspectral soil reflectance measurements and their transformations, while predicting the percentages of sand, clay and SOM content, in situ as well as in laboratory conditions.

2. Materials and Methods

2.1. Data Collection

Three fields at McGill University Macdonald research farm (45°24′42.2″ N 73°56′23.1″ W; Sainte-Anne-de-Bellevue, QC, Canada) were used for this project. In total, nineteen locations were selected in these fields (Figure 2) to represent diverse soil conditions ranging from sand to clay loam. The fields had 2.5, 4.5 and 12 ha areas and crops were grown according to a rotation involving soybean, corn (grain and silage) and alfalfa. All data were collected in 2012 when an alfalfa/grass mix was grown in each of the three fields.

In each location, composite soil samples were obtained from the top (0–20 cm) layer of soil. A 4 cm diameter, stainless steel auger was used to take five samples from within a 0.5 m radius. These soil cores were mixed, air dried, and sieved through a 2 mm mesh. Then, each sample was divided into two subsamples: one to be used for conventional laboratory analysis and the other for ex situ vis–NIR spectral measurements.

Three categories have been established for soil particles: sand, silt, and clay [55]. These three groups are called soil separates. The three groups are divided by their particle size. Clay particles are the smallest, while sand particles are the largest. Particle size analysis (fractions of sand, silt, and clay) as well as SOM content were evaluated for each laboratory sample, as summarized in

Table 1. Statistical results of the percentages of sand, clay, and SOM content in 19 soils.

Statistics	% Sand	% Clay	% SOM
Min	26	2	3.9
Median	56	9	5.8
Max	93	33	25.6
Mean	56	13	7.2
SD	18	9	4.8

. The particle size analysis was conducted using hydrometers [1] and SOM was determined using the loss-on-ignition technique [2].

The core of the Veris P4000 instrument is a combined dual type spectrophotometer operating in the visible and near-infrared parts of the electromagnetic spectra (Figure 3). One of the two spectrophotometers was USB2000 (Ocean Optics, Dunedin, FL, USA) operating between 342 and 1023 nm with a spectral resolution of 6 nm. The other spectrophotometer was C9914GB (Hammatsu Photonics. K.K., Tokyo, Japan), which collected data between 1070 and 2220 nm with a spectral resolution of 4 nm. The instrument included its own light source and was capable of maintaining a constant distance between measured soil surfaces and detectors by means of a sapphire contact probe with fibre-optic cables.

All ex situ measurements were conducted as triplicates in a different randomised order for each replicate. A specially designed sample holder (Figure 3b) was filled with <1 g of the soil sample and placed near the optical window of the soil profiling tool. As was recommended in the user manual provided by the manufacturer, at the beginning of each spectral measurement session, the instrument was calibrated by measuring the dark current followed by the white reference measurements using the specially provided reference blocks, as shown in Figure 3c. The instrument was re-calibrated after every 20 samples. Soil spectra were interpolated to about 5 nm of the spectral resolution, yielding a total of 380 data points (wavelengths) per spectrum. To minimize the instrument noise, each spectrum was the average of 25–30 scans (~6 scans s⁻¹). The in situ measurements were collected using the recommended equipment setup (Figure 4) for topsoil profiles down to 20 cm, while penetrating soil at a speed approximately 2 cm s⁻¹. Three measurements were conducted consecutively along a straight line that was less than 0.5 m long. The instrument was re-calibrated in a similar manner as discussed for the ex situ measurements. The soil spectra data collection procedures for in situ and ex situ measurements were the same. However, in this case, the average of 50–60 scans represented soil spectra collected at different depths from 0 to 20 cm.

2.2. Data Processing

As discussed previously (in Section 2.1) and as shown in Figure 5, the DRS measurements were collected in triplicate on representative soil samples using the soil profiling tool (ex situ and in situ), whereas the representative soil samples were analyzed in laboratories to obtain the standard measurements of the soil attributes of interest. Next, the DRS measurements were processed to create three more data sets (four in total: Raw, Smooth, 1st SGD, and 2nd SGD). Later, to understand the random fluctuation of soil spectra within the same samples and between different samples, the spread (SD_SA) and the root mean squared deviation (RMSD) were calculated on the DRS measurements for all four data sets. Simultaneously, the best SLR and PLSR models were fit on all four DRS measurement data sets against the known standard measurements of the soil attributes of interest. Finally, while focusing on the reproducibility of soil reflectance and its direct link with soil attributes, precision- and accuracy-related components such as the ratio of spread over error (RSE), measurement precision (MP), standard error of prediction (SEP), and coefficient of determination (R²), respectively, were calculated. These are explained in greater detail later in the text.

All raw spectral data were processed using MATLAB 2012a (The MathWorks, Inc. Natick, MA, USA) and ParLeS 3.1 software (University of Sydney, Sydney, Australia), as described by [56]. All spectra exhibited a step discontinuity from 1023 to 1070 nm, caused by the transition from one detector to another. After removing the relatively noisy parts of the spectra at the edges of the detection ranges for each spectrophotometer (342–409 nm, 1014–1075 nm, and 2206–2220 nm), all resultant spectra consisted of a total of 363 measurements at different wavelengths. The spectral data were then corrected for offset and processed using a multiplicative scattering correction (MSC) algorithm [57] and mean centering (MC). In addition to these “raw” spectra, the following spectra treatments were pursued: (1) 3-point Savitzky–Golay smoothing, (2) 11-point first order Savitzky–Golay derivative (1st SGD), and (3) 11-point second order Savitzky–Golay derivative (2nd SGD) [58].

Among the different statistics considered for assessing the precision of spectral data, the RSE was used:

$$RSE=\frac{S{D}_{SA}}{RMS{D}_{}}$$

(1)

where SD_SA is the standard deviation of nineteen sample mean values; RMSD is the root mean squared deviation calculated based on three replicated measurements for each of the nineteen samples.

$$S{D}_{SA}=\sqrt{\frac{{\displaystyle \sum _{i=1}^n(X_i}-\overline{X}{)}^2}{n-1}}$$

(2)

where $n$= 19 is the number of soil samples; $X_i$ is an average measured or calculated spectra related value for the ith sample; $\overline{X}$ is the average of all $X_i$ values.

$$RMS{D}_{}=\sqrt{\frac{{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{j=1}^m{\left(x_{ij}-X_i\right)}^2}}}{n\cdot \left(m-1\right)}}$$

(3)

where $m$ = 3 is the number of replicated measurements; $x_{ij}$ is a jth replicate of the measured or calculated spectra-related value for the ith sample

$$X_i=\frac{{\displaystyle \sum _{j=1}^m{x}_{ij}}}{m}$$

(4)

$$\overline{X}=\frac{{\displaystyle \sum _{i=1}^n{X}_i}}{n}$$

(5)

Similar to the frequently used ratio of prediction over deviation (RPD) [59], high RSE means a relatively strong ability of a given measurement to distinguish different soil samples. The use of RSE was reported earlier in [60,61] and is directly related to ANOVA F statistics used to compare the means of repeated measurements. Based on the degrees of freedom involved, the difference among the soil samples (means of three measurements) can be detected at α = 0.05 when RSE is greater than 0.79 ($\sqrt{\frac{1}{m}{F}_{stat}}$). This analysis evaluates measurement precision with the underlying hypothesis that a particular parameter that does not change when measuring the same soil sample, and for which the change is at its maximum when measuring different samples, should be considered reliable.

The percentages of sand, clay, and SOM were predicted by fitting SLR models on each individual measured or treated spectral value versus these properties. A coefficient of determination (R²) was the main indicator of the ability of a single spectral value to explain the variability in a particular soil property. However, the standard error of prediction (SEP) was used as a measure of the accuracy of soil property estimates obtained using each SLR model:

$${\widehat{y}}_{ij}={\beta }_0+{\beta }_1{x}_{ij}$$

(6)

where $x$ is the measured value of a given soil property for the ith sample; $y$ is the predicted value of a given soil property for the ith sample and the jth replicate; ${\beta }_0$ (intercept) and ${\beta }_1$ (slope) are coefficients of SLR.

$$SEP=\sqrt{\frac{{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{j=1}^m{\left(y_i-{\widehat{y}}_{ij}\right)}^2}}}{n\cdot m}}$$

(7)

where y is the measured value of a given soil property for the ith sample.

While SEP indicated the total error associated with each individual measurement and is primarily linked to the accuracy of prediction, RMSD is linked to measurement precision. However, since RMSD is expressed in the units of spectra measurements and related calculated parameters, measurement precision (MP) can be expressed in physical units as follows:

$$MP={\beta }_1\cdot RMSD$$

(8)

When comparing different spectral wavelengths and transformation techniques, it is important to identify measurements that have the maximum RSE and the minimum RMSD, MP, and SEP. The RMSD is an indicator of measurement reproducibility. However, without considering the spread of values across different samples, it is impossible to conclude if the given values are strong values to distinguish different soil samples from each other. Therefore, the RSE is involved in electing candidates to differentiate between the samples’ levels of disregard of the prediction property. Neither RMSD nor RSE depend on the model used to predict a given soil property.

Because of the SLR approach [62] used to test one-input soil property prediction functions, RMSD can be expressed, in terms of the percentages of sand, clay, or SOM, as MP. The MP estimate is then evaluated together with SEP, which is the ultimate indicator of the predictability of soil properties. Unlike RMSD, the MP as well as the SEP can be compared across different spectra transformation procedures as both are expressed in physical units. From a sensor development point of view, a small RMSD and MP indicates a stable soil–detector interface. A high RSE means that the sensor can be applied to a particular set of soils. Finally, a small SEP (high R² for a given set of samples) indicates the sensor’s ability to predict the soil property of interest. The SEP is always greater than the MP, and the greater this difference, the less uncertain the linear relationship between the measured value and the property. Small differences between the SEP and MP indicate the applicability of the prediction model when reliable measurement estimates are obtained. In other words, small differences between the SEP and MP indicate the potential for improved predictability by averaging multiple unbiased measurements, but larger differences signify the limitations of the model and that alternative prediction methods, such as PLSR, should be involved.

In soil spectroscopy, PLSR is one of the most widely used techniques to aggregate measurements obtained at multiple wavelengths in a single prediction model [32,33,34,35] The PLSR is a bilinear regression technique that extracts a small number of latent factors, which are a combination of the independent variables, and uses these factors as a regression producer for the dependent variables [63,64]. The PLSR analysis is normally evaluated using the leave-one-out cross validation technique, and the RMSD, R² and Akaike Information Criterion (AIC), as described in [65], are the most common model performance indicators.

In this study, the orthogonalised PLSR-1 algorithm described in [66] was applied to (1) raw spectra, (2) smoothed spectra, (3) 1st SGD spectra, (4) 2nd SGD spectra, and (5) all the values were combined to develop calibration models using ParLeS software [56]. The number of factors to use in each model was selected using leave-one-out cross validation. However, due to the limited number of soil samples used ($n=19$), the selected models were not re-validated on a different set of soil samples; however, this was generally undertaken in other reported studies. The developed models were used to estimate the performance indicators that were comparable to MP and SEP to define the superiority of PLSR over SLR models.

3. Results and Discussions

Statistics derived from laboratory results based on the percentages of sand, clay and SOM contents are listed in Table 1. These results demonstrate the variability observed over relatively small fields in terms of their texture and SOM content. It is interesting to note that the standard deviation of % sand in this dataset is two times higher than that of % clay and four times higher than that of % SOM.

Average reflectance spectra of nineteen soil samples, collected both ex situ and in situ, are shown in Figure 6. In general, the laboratory spectra exhibited higher reflectance when compared with those from the field, which might be explained by dry versus wet soil. All spectra showed water absorption inversions near 1400 and 1900 nm [24,38]. Less noticeable inversions were also observed in many other parts of the spectra (1100, 1500, 1600, 1700, and 2100 nm) which could be associated with the primary or secondary effects of various minerals, carbonates and SOM [24,38]. Several noticeable peaks were also observed at certain parts of the spectra, especially in the visible region, that could be associated with soil color.

Figure 7 illustrates the RSE values produced for the raw and all transformed spectra and

Table 2. Ratio of spread over error (RSE) comparison.

Test	Transformed Factors	Min	Median	Mean	Max
Ex situ	Raw	1.69	1.76	1.81	2.18
	Smooth	1.62	3.04	3.37	8.71
	1st SGD	0.47	1.91	2.49	6.93
	2nd SGD	0.32	0.86	1.32	10.56
In situ	Raw	1.24	1.53	1.51	1.67
	Smooth	1.38	2.23	2.35	4.30
	1st SGD	0.89	2.14	2.33	5.34
	2nd SGD	0.52	1.59	1.70	3.88

summarizes the relevant statistical results. It is noticeable that transforming the raw soil spectra into a 3-point Savitzky–Golay smooth spectrum, and the first and second order Savitzky–Golay derivative spectra, increased the reproducibility in many parts of the spectra. It is clear that the second SGDs had the highest relative reproducibility across certain parts of the ex situ spectrum, while RSE for the first SGDs had the highest relative reproducibility across certain parts of the in situ spectrum. One might expect that ex situ measurements were more reproducible than those conducted in natural soil conditions within a 0.5 m distance from each other. However, in both cases, first SGD yielded data with comparable relative reproducibility in certain parts of the spectrum. In large part, each transformed spectra had some portions with the ability to separate the means of the three replicated measurements between different soil samples (RSE > 0.79).

Figure 8 illustrates the coefficients of determination versus the RSE’s, while predicting the percentages of sand, clay, and SOM contents using SLR.

Table 3. vis–NIR wavelengths (nm) indicating the highest correlation (R²) and ratio of spread over error (RSE) and with % sand, % clay and SOM content.

Soil Property	Test	Raw	Smooth	1st SGD a	2nd SGD b
% sand	Ex situ	-	-	1932, 1936, 1940	1413
	In situ	-	-	1447, 1452, 1940, 1944	1457, 1462, 1466, 1923
% clay	Ex situ	-	-	1936, 1940, 1944, 1948	1433, 1915, 1919, 1923
	In situ	-	-	1931, 1936, 1940, 1944	1919, 1923
% SOM	Ex situ	733, 786	712, 744	2114, 2117, 2121, 2148	-
	In situ	-	749, 770	-	602, 1738, 1742, 2159, 2182

^a First order Savitzky–Golay derivative. ^b Second order Savitzky–Golay derivative.

summarizes the wavelengths and transformation methods indicating the highest level of predictability for the soil properties of interest. Selection of these wavelengths was predominantly guided by the coefficient of determination as the weighting between R² and RSE is arbitrary.

Alternatively, Figure 9 illustrates SEP versus MP while predicting the soil properties. All values were calculated using both the SLR and PLSR calibration models. To facilitate the comparison, the standard deviation of soil properties (laboratory measurements) was shown as a benchmark for SEP and the isolines represented positions in the chart with the constant sum of MP and SEP. To assure that both error terms are small, the most promising wavelengths and spectra transformation methods were selected as well as those placed between two isolines, indicating the lowest sum of MP and SEP.

Table 4. vis–NIR wavelengths (nm) indicating the lowest critical error’s when predicting % sand, % clay and SOM content.

Soil Property	Test	Raw	Smooth	1st SGD a	2nd SGD b
% sand	Ex situ	-	-	1931 c, 1936 c, 1940, 1944	1413 c
	In situ	-	-	1452, 1457, 1931,1936, 1940, 1944	1462 c, 1923
% clay	Ex situ	-	-	1940, 1944, 1948	1915, 1919, 1923
	In situ	-	-	1931, 1936, 1940 c, 1944c	1923
% SOM	Ex situ		674 c, 679, 685	551, 556 c, 562, 568, 585	-
	In situ	-	421, 722, 728	421, 585, 590, 1747 c, 1751	1605, 1543

^a First order Savitzky–Golay derivative. ^b Second order Savitzky–Golay derivative. ^c vis–NIR avelengths that are common between in situ and ex situ measurements.

summarizes these wavelengths and spectra transformation methods.

Table 5. The vis–NIR soil spectral bands which were capable of exhibiting the highest possible degree in terms of the precision and accuracy components, when predicting % sand, % clay and SOM content during ex situ measurements.

Soil Property	Spectra	Model	Wavelengths, nm	RSE a	R2	MP b	SEP c
% sand	2nd SGD d	PLSR f	-	-	-	2.96	3.78
	1st SGD e	SLR g	1931	4.26	0.59	3.22	11.31
	1st SGD	SLR	1936	4.77	0.58	2.87	11.40
	2nd SGD	SLR	1413	3.58	0.56	3.70	11.74
	1st SGD	SLR	1940	5.06	0.56	2.65	11.75
	1st SGD	SLR	1944	5.57	0.50	2.29	12.46
% clay	1st SGD	PLSR	-	-	-	0.63	0.73
	1st SGD	SLR	1944	5.57	0.87	1.59	3.33
	1st SGD	SLR	1940	5.06	0.87	1.74	3.35
	2nd SGD	SLR	1920	9.06	0.86	0.98	3.55
	2nd SGD	SLR	1924	10.38	0.85	0.85	3.61
	2nd SGD	SLR	1915	8.07	0.83	1.08	3.86
% SOM	2nd SGD	PLSR	-	-	-	0.99	1.09
	1st SGD	SLR	557	4.24	0.36	0.67	3.78
	1st SGD	SLR	551	4.86	0.35	0.59	3.79
	1st SGD	SLR	585	4.26	0.34	0.65	3.84
	1st SGD	SLR	563	4.46	0.33	0.62	3.86
	Smooth	SLR	685	6.70	0.31	0.40	3.92
	1st SGD	SLR	568	4.89	0.31	0.54	3.93
	1st SGD	SLR	680	7.35	0.30	0.36	3.95
	Smooth	SLR	674	7.67	0.29	0.34	3.99

^a Ratio of Spread over error. ^b Measurement precision. ^c Standard error of prediction. ^d Second order Savitzky–Golay derivative. ^e First order Savitzky–Golay derivative. ^f Partial least squares regression (PLSR) calibration model. ^g Simple linear regression (PLSR) model.

and

Table 6. Summary of vis–NIR factors demonstrating reproducibility, precision and accuracy during in situ measurements.

Soil Property	Spectra	Model	Wavelengths, nm	RSE a	R2	MP b	SEP c
% sand	Raw	PLSR f	-	-	-	2.64	4.09
	2nd SGD d	SLR g	1462	1.90	0.49	4.25	10.86
	1st SGD e	SLR	1452	3.75	0.40	3.93	11.67
	1st SGD	SLR	1944	2.34	0.50	3.96	11.83
	2nd SGD	SLR	1924	2.17	0.39	3.35	12.06
	1st SGD	SLR	1940	2.74	0.56	3.54	12.13
	1st SGD	SLR	1936	3.30	0.58	3.31	12.33
	1st SGD	SLR	1457	4.31	0.36	3.50	12.35
% clay	1st SGD	PLSR	-	-	-	1.28	1.73
	1st SGD	SLR	1940	2.74	0.87	2.41	3.17
	1st SGD	SLR	1936	3.30	0.85	2.28	3.38
	1st SGD	SLR	1944	2.34	0.87	2.58	3.75
	1st SGD	SLR	1932	3.64	0.78	2.32	4.28
	2nd SGD	SLR	1924	2.17	0.85	2.12	4.46
% SOM	1st SGD	PLSR	-	-	-	1.31	2.05
	1st SGD	SLR	1747	3.07	0.00	0.86	3.46
	2nd SGD	SLR	1605	1.67	0.01	0.96	3.57
	1st SGD	SLR	1752	2.67	0.00	0.85	3.59
	1st SGD	SLR	1544	1.39	0.00	0.96	3.59
	Smooth	SLR	422	2.77	0.28	0.74	3.76
	Smooth	SLR	728	3.87	0.33	0.72	3.78
	Smooth	SLR	723	3.97	0.34	0.69	3.82
	1st SGD	SLR	591	4.60	0.00	0.48	3.98
	1st SGD	SLR	422	2.80	0.00	0.51	3.99
	1st SGD	SLR	585	5.07	0.00	0.52	3.99

^a Ratio of Spread over error. ^b Measurement precision. ^c Standard error of prediction. ^d Second order Savitzky–Golay derivative. ^e First order Savitzky–Golay derivative. ^f Partial least squares regression (PLSR) calibration model. ^g Simple linear regression (PLSR) model.

list key model performance indicators for each variable of prediction listed in Table 4 along with all PLSR models. Figure 10 points out selected wavelength using the average of all raw spectra.

It was observed that wavelengths near 1900 nm (short water absorbance band) transformed using either the first or second order derivative were found to be better correlated with the percentages of sand and clay content as compared to other parts of the spectra. Smooth and first SGD spectra corresponding to different visible wavelengths were superior when predicting SOM content.

From Table 5 and Table 6, it is obvious that PLSR results had the lowest combination of MP and SEP with no overwhelming difference among the types of spectra transformations involved. In most cases, PLSR that was based on second SGD spectra indicated the best performance and the prediction soil properties did not improve when using all four methods of spectral transformation. This was an expected observation as PLSR uses several spectral factors and SLR just one; therefore, it is not capable of relating the difference in sensor response to a given soil property.

As illustrated in Figure 10 (and also summarized in Table 5 and Table 6), when SLR models were applied to ex situ measurements, the best predictor wavelengths (i.e., single variable predictor while considering the sum of both precision (MP) and accuracy (SEP)) were found to be 1936 nm and 680 nm to correspond well with % sand and % SOM, respectively. However, the measurements needed to be transformed into the first SGD, whereas 1924 nm corresponded well with % clay only after transforming the measurements into the second SGD, which was similar to the results obtained when SLR models were applied to in situ measurements.

Then, the best predictors’ wavelengths (i.e., single variable predictors while considering the sum of precision (MP) and accuracy (SEP) were found to be both 1940 nm and 1747 nm, which corresponded well with % clay and % SOM, respectively. However, the measurements needed to be transformed into the first SGD, whereas 1462 nm corresponded well with % sand only after transforming the measurements into the second SGD.

In general, the results indicate that when using a single predicting factor, the SEP for sand was about 10–12% with no significant difference between in situ and ex situ measurements. The MPs of around 3% indicate relatively strong spectra reproducibility, which indicates the need for a more expanded model. This is apparent since the SEP of PLRS was two times smaller, indicating the ability to predict sand within a margin of 4%.

The percentage of clay, on the other hand, had 3–4% SEP and 1–2% MP, indicating both the reproducibility of the spectra and the ability of a single spectra parameter to accurately predict clay. In this case, PLRS also reduced the errors by two times, but only using first, or second, order derivative spectra.

Finally, the SEP for SOM was only a quarter lower than the standard deviation obtained in laboratory measurements [67], indicating that SLR is not an appropriate model for this soil property for the given set of soils. PLSR reduced SEP to about 1% SOM, which is reasonable considering the extensive spread of soil texture in this soil set. Similar to sand and clay, minor increases in error estimates corresponding to in situ versus ex situ measurements were much smaller than expected considering that different physical locations within a 0.5 m distance were used each time.

Altogether, these results are comparable with other reports pertaining to the use of DRS for the measurement of soil texture and SOM, as previously summarized in [6,16,19,20,24]. The relatively small number of locations means that the results for PLSR are rather optimistic and, once the multivariate regression approach is applied, the expected results should be between the present PLSR and the best SLR performance indicators. However, the most important finding in this study is that the majority of spectral measurements are sufficiently reproducible to be considered for distinguishing among diverse soil samples, while certain parts of the spectra indicate the capability to achieve this at α = 0.05.

Since SEP was typically at least two times greater than MP, it is clear that a simplistic SLR modeling method does not account for a substantial part of variability among different soil samples; this might be explained by the interactions among different soil properties as well as the fact that different soil properties cause different degrees of change in specific spectral parameters. This would imply that a suitable prediction model should involve multiple such parameters, yet their number should be relatively small to assure model applicability outside a given set of soil samples. This approach is appropriate when analyzing other soil properties and PSS systems geared to investigate the spatial heterogeneity of specific soil attributes. However, the goal and the purpose of this paper was to use the simplest method that focused on the reproducibility of soil reflectance spectral elements and to assess direct relationships with soil attributes rather than model development [38,39,40,41,68]. The latter is the subject of follow-up work.

4. Conclusions

Based on the results of this study, it can be concluded that vis–NIR spectral measurements obtained using the commercially available portable equipment are compatible in terms of precision and accuracy when they are used under laboratory conditions or in the field, and when soil water content varies substantially. Data processing by performing smoothing, and first and second order derivative transformation, of the spectra has increased measurement reproducibility, which enhanced the system’s ability to separate different soil samples. This was clearly noticeable after applying the innovative RSE calculation method on each individual spectra and its smooth or derivative transformations. Naturally, this indicated the ability to predict soil properties using a simple SLR technique that defined the difference between soil samples. From the soil attributes tested using the SLR method, clay content was shown to be predictable using spectral data derivatives around the short water absorption band. The prediction error was higher for sand and unacceptably high for SOM. Use of the popular but complex PSLR technique reduced prediction errors by between a quarter and one third, and the results were found to be comparable with previous studies pertaining to the use of DRS for measuring soil texture and SOM. However, a relatively small number of soil samples require caution when inferring results in practice. The most important finding in this study is that the majority of vis–NIR spectral measurements are sufficiently reproducible to be considered for distinguishing among diverse soil samples, while certain parts of the spectra indicate the capability of achieving this at α = 0.05. The methodology for evaluating both the precision and accuracy components of spectral measurement errors was found to be useful when assessing the performance of a given proximal soil sensing instrument.