New Approach for Photogrammetric Rock Slope Premonitory Movements Monitoring

Abstract

An automated, fixed-location, continuous time-lapse camera system was developed to analyze the existence of rockfall precursory movements and quantify volume changes after detachments. It was implemented to monitor the basaltic formation on which Castellfollit de la Roca village is built. Due to the geometrical conditions of the area, the camera system consists of three digital cameras managed by a control unit that contains a Raspberry Pi 4 microprocessor. Images taken every day are sent to a server for processing. A workflow has been developed to work with a set of images with an irregular time interval to detect precursor movement. The first step consists of matching the images with a reference master image and filtering the vegetation to improve the process using a mask obtained by a green leaf index (GLI) index. Then, the adjusted images are used for a forward-backward correlation process carried out to detect movements. If movement is detected, a 3D model is built using structure from motion (SfM) to quantify the movements. The system has been working since September 2021. During this period, movements from 0.01 to 0.5 m and several rockfalls of a small volume have been detected.

1. Introduction

Rockfalls are predominant processes in mountainous areas. This term applies to blocks or rock masses that, after separation from a steep wall, experience free fall and subsequent impact against the ground, with the consequent rebound, possible rolling and/or sliding of rock fragments [1,2]. These phenomena have great destructive potential due to the high velocities, and thus high kinetic energy, that blocks may reach during their propagation [3]. Buildings, civil infrastructures and transportation networks are often threatened by rockfalls. A recent study showed they cause the greatest number of victims due to slope instability in Spain [4]. This study observed a reduction in fatalities due to rockfalls over time due to awareness and protection measures. However, since the dynamic coupling between global warming and mountain-slope instability has increased significantly [5,6], an increment in the number of events may be expected.

Monitoring plays an important role in managing risk due to rockfalls in a wide spectrum of applications. To inventory events that occur, terrestrial or aerial laser scanning (TLS or ALS) and photogrammetry are frequently used to obtain surface models of the terrain, the position of the deposited block fragments, an estimation of the detachment volume, the fragment trajectories, etc. [7,8]. These techniques are used to quantify detached rock masses on rock slopes over time to estimate the expected magnitude frequency of rockfall [9,10,11,12,13,14], which is required to evaluate risk quantitatively [15,16]. For early warning purposes, ground-based radar interferometry [17], seismic signals [18] and acoustic signals [19] have proven useful in a variety of hazardous scenarios. For better understanding of the rockfall process, some authors have recently used videogrammetric techniques to track blocks and fragments generated during breakage in controlled scenarios on real-scale tests [20,21] or laboratory tests [22,23] and seismic analysis techniques [24,25].

In rockfalls, premonitory movements tend to be very small, even in certain cases of the same order of magnitude as instrumental errors [26,27], and, in some scenarios, small detachments may be premonitory of a larger event [28]. These small movements, together with the need for high temporal continuity, led us to select terrestrial photomonitoring for this application. Satellite techniques were discarded due to the verticality of the rock slope. A continuous or semi-continuous approach was required for this application, but some of the leading techniques, like LiDAR or ground radar, were discarded due to their high cost.

Some authors have already tested photomonitoring devices on rock slopes. For example, ref. [29] collected a workflow and a series of recommendations for the correct use of photogrammetry with time-lapse type cameras for dynamic geological studies, such as the detection of changes in slopes, monitoring of movements and changes in roughness. This type of photogrammetry costs less than other measurement techniques (such as TLS). The authors showed the importance of environmental conditions (lighting, etc.). (Westoby et al., 2012) [30] provided an example of a coastal cliff and how the SfM (structure from motion) technique is adequate from a technical and economic point of view and is efficient and flexible enough to capture complex topographies. In Blanch et al. [10], smoothed models and the application of basic statistics (of the median of a data set) were used. They calculated a digital terrain model (DTM) from the median of n low-resolution DTMs (low-cost system). However, the detection of premonitory movements using this photomonitoring still has room for improvement.

The aim of this study was to evaluate the viability of detecting premonitory movements with a low-cost monitoring system in rock slopes with a continuous low-cost photomonitoring system using currently available open-source hardware and software for data capture. For image processing, we have implemented our own software. In this article, we describe the design and performance of a time-lapse fixed position system with three digital single-lens reflex (DSLR) cameras to monitor a basaltic cliff. The image process system can detect small precursor movements of around a centimeter. Although, this premonitory movements detection system is focused mainly on the existence of this displacement more than its magnitude.

The analysis is based on a comparison of the three synchronized images at a specific time, t, with images of an initial time, t₀. If any change greater than one pixel is detected, an alert is “sent”. Then, the Digital Surface Model (DSM) is generated automatically, with knowledge of some ground control points. Due to the fact that this system measures continuously, it allows the detection of all the fallen blocks, as well as their volume. This information allows updating the temporal evolution of the massif due to the changes in the annual probability. This information, which is often ignored, is useful for the dynamic analysis of the risk [31].

Castellfollit de la Roca was chosen for this study for several reasons. One reason is the social interest of the area, as the population center is situated on top of the basaltic formation, and therefore the evolution of the rock massif must be controlled due to the danger it could pose to the population. In relation to the previous point, this is an active area. This has been shown by various campaigns using TLS and photogrammetry from unmanned aerial vehicles (UAVs) that have been carried out by the Institut Geològic i Cartogràfic de Catalunya (ICGC) [32]. Therefore, the volume loss results can be compared.

The previous campaigns have shown that rockfalls occur. However, due to the nature of the campaigns and the methodology used, it is difficult to quantify whether precursor movements happen. This can only be determined using a continuous system. To avoid the problems of the area’s geometry, which are similar to those of other areas, we only worked with images to detect movement. This is unlike the study by the other authors. Notably, the geometry of a camera shot influences the precision in the measurement of the points in the photogrammetric model that is generated. The device that is designed will work with three cameras to minimize and guarantee data redundancy.

2. Methodology

2.1. Study Site

The study area is in the village of Castellfollit de la Roca (Catalunya, Spain) at an altitude of 296 m. It is located at the top of a Quaternary basaltic north-facing cliff face that is longer than one kilometer. The place is of great height. The distance between the lower part on the banks of the Fluvià river to the upper part of the town is about 40 m. There is abundant vegetation in the area, both on the wall, where the basaltic columns have shrub vegetation, and on both banks of the river, where there are a large number of tall trees.

This basaltic formation was made by erosive action of the rivers Fluvià and Toronel on the volcanic remains resulting from the superposition of two lava flows, Figure 1. These lava flows are estimated to be 217,000 and 192,000 years old, respectively. On this cliff, prism-shaped disjunctions that are formed when the basalt cools slowly and at rest are present. Due to erosion from the cliff base, several columns have collapsed in recent centuries. There are two kinds of failure: toppling and vertical falling by gravity. Abellán et al. [33,34] carried out observation campaigns with TLS and detected in this area precursory deformation of the rockfall lower than 1.64 cm/year, followed by an apparently stationary stage.

Usually, the risk in rockfall is due to the fact that the exposed element that is at risk is under the cliff. In this particular case, it is on top of the cliff. The erosion of the massif on which the town is built can put at risk the buildings that are closer to the edge. This peculiarity is the reason why we chose this area, focusing on the closest part of the cliff to the buildings and the most active part of the cliff, according to previous monitoring campaigns.

2.2. System Design

For the photogrammetric capture, a minimum set of photographic cameras (three cameras) was used. They were located at an approximate distance of 37 m between the camera in the center and those at the ends, with convergent image capture. All cameras were visible to the central zone of the model to control. With this configuration, the overlap between images was close to 90% among all of them. Figure 2 shows the area covered by each camera: in red camera 1, in green camera 2 and in blue camera 3.

One of the most important data capture factors that influence the results is the optical characteristics of the cameras [35]. Bearing in mind the geometric conditions of the area, different models were analyzed, selecting the Canon EOS 6D Mark II model, which has a CMOS-type sensor with a 3/2 ratio, a size of 35.9 × 24.0 mm, resolution of up to 6240 × 4160 pixels, and a 105 mm focal length. Fixed photographic conditions were used to take the photographs, such as sensitivity at ISO 100 and an aperture of f/8, to leave the calculation of the exposure time based on the fixed variables and the reading of the camera’s photometer.

The black color of the basaltic cliff, the position and the vertical structure of the prisms cause different problems in taking the photographs and in the photogrammetric processing, such as unwanted brightness, homogeneity in the surfaces, cracks and damp surfaces, and rapid growth of seasonal vegetation that has to be filtered before any other process.

Due to these conditions, especially the morphological ones, it was necessary to move a significant distance away to take the photographs to ensure that the inclination of the cameras and the difference in scale in the same photograph was minimal. Finally, it was decided to fix the photographic cameras in the industrial buildings near the riverbank at a distance of approximately 170 m, which is greater than in other studies [10,35,36,37]. As mentioned above, the cameras were fixed at a similar distance, in stable places, and in a line almost parallel to the projection of the cliff in the study area. The relationship between them and the photogrammetric model gave an average scale of 1/1700, with scale variations of less than ± 10%.

Once the cameras and their localization had been selected, the other important factor was solving the technical requirements to build a system that could remotely take pictures at regular intervals and send them to the office by internet connection [38]. Due to the option of connecting this system to the public electricity network, we avoided the need for solar panels and accumulation batteries. However, this connection was essential to control possible electrical spikes using an electrical transformer to filter them and a surge protection system to protect all components. All these components were enclosed in an IP67 cage to ensure water protection, and they constituted what we called the power management unit.

The last component of the system is the control unit, whose mission is to send the order to take a photograph and then send the collected pictures over a 3G network to a server. The control unit functions with a Raspberry Pi 4 (RPi4) microcomputer, which is capable of interfacing through USB protocol with the cameras that are directly wired to it. Since the distance to the cameras was longer than 10 m, supported by the USB protocol, a USB extension through ethernet was required for cameras C1 and C3. Moreover, it had an Rj45 connector for internet connectivity. Finally, a relay board connected to the RPi4 controlled the camera power supply and thus when they were turned on or off.

With this low-cost system, periodic photogrammetric coverage towards the front of the cliff was carried out.

2.3. Error Analysis in 3D Photogrammetric Models

We can estimate the errors in the measurement of the coordinates at a point in the photogrammetric model, considering only factors such as the geometry of the set cameras-object, the relationship between the photographic base (practically parallel to the wall), the distance to the object of the 1/5 type and, finally, the characteristics of the cameras. These errors can be determined from the scale ratios and parallax equations [39], as shown in Equations (1) and (2).

$$E=\frac{Y}{f}=\frac{B}{p}$$

(1)

where $E$ is the scale factor, $Y$ the distance from reference to observed point, $B$ the photographic base, $p$ the parallax, dp = $\sqrt{2}dx$ and $f$ the focal length.

$$X=x·E\hspace{1em}\hspace{1em}Y=f·E=\frac{B·f}{p}\hspace{1em}\hspace{1em}Z=y·E$$

(2)

$$dX=dx·E\hspace{1em}\hspace{1em}dY=\frac{Y^2}{B·f}dp\hspace{1em}\hspace{1em}dZ=dy·E$$

(3)

where $X,Y,Z$ are the ground coordinates (GC), $x,y$ the image coordinates, $dX,dY,dZ$ are the estimated error in GC and $dx,dy$ the error image coordinates. All the variables are expressed in meters, although usually, the image coordinates and focal length are originally in mm.

Applying Equation (3), we can calculate the terrain variation that we can observe from the image coordinates. Considering an appreciation in the identification of the point on a target to be measured of half a pixel (dx = dy = 1/2 px) and the least favorable case, i.e., working with only two contiguous cameras and, therefore, with a lower base-to-object ratio and a major value of E, we obtain values for the movements in terrain that we could appreciate, $dX,dY$ and dZ. These would be 0.5 cm in X and Z and 3 cm in depth (Y coordinate). In the case of identifying points on natural details, the appreciation would be one pixel, and the detectable displacements would be double.

Moreover, the workplace is influenced by different natural circumstances that can modify the theoretical estimation, such as artificial or direct light on the camera lenses and excessive humidity in the area. Another difficulty is the unequivocal identification of tie points, whose coordinates were obtained by topography. These two factors incorporate a degree of uncertainty that is difficult to evaluate from theoretical expressions. Therefore, a test was carried out to check that the estimated errors were really those expected. In this test, two targets were displaced by a known amount. The objective was to check the real discrepancy between the theoretical values (imposed in the field) and those obtained in the photogrammetric process.

A guide was designed to control the displacements of the targets. This consisted of a horizontal guide parallel to the coordinate system, and another guide perpendicular to it, as shown in Figure 3. This made it possible to measure displacements in the X-Z plane (H) and in-depth D (Y axis). Two targets were placed on these guides, which were first taken at the origin of the guide and then moved to 2 cm, 5 cm and 10 cm, and in the case of depth, also to 20 cm, as shown in

Table 1. Discrepancies between the real displacement and the displacement measured in the photogrammetric model in the X-Z plane (H) and in-depth (D). Units in cm.

Real Displacement (H; D) cm	Horizontal Displacement Measured (H) cm	Discrepancies (H) cm	Depth Displacement Measured (D) cm	Discrepancies (D) cm
0; 0	0.0	0	0.5	0.5
2; 2	2.0	0	2.5	0.5
5; 5	5.0	0	5.3	0.3
10; 10	9.7	0.3	10.4	0.4
0; 20	---	---	18.7	1.3

. Moreover, fixed reference targets were used for checking.

The software used for the photogrammetric adjustment to obtain the models and the coordinates of the points of interest was Agisoft Metashape Professional, Version 1.8.1 (Agisoft LLC., St. Petersburg, Russia). The block adjustment of all the photographs was carried out in packs of six images per position. Since the images were taken at short time intervals, the atmospheric and camera geometry conditions were considered to be the same. Therefore, the same calibration values were used for all of them. The average ground photographic resolution was 1 cm/px, and the reprojection error achieved in the photogrammetric process was less than 1 px.

The discrepancies between the real displacements (from origins at 0, 2, 5, 10 and 20 cm) and those obtained from the photogrammetric model are shown in Table 1. The maximum discrepancy of 0.3 cm on the horizontal displacement occurs at a distance of 10 cm, while for lower values, expected for the movement of the columns, there was no discrepancy. As expected, in-depth, this discrepancy was larger, at about 0.4 mm on average, and occurred in all cases in a fairly uniform way, except for the value of 20 cm when the discrepancy rose to 1.3 cm. These values were consistent with the theoretical calculations since we were working in the most favorable case with the three cameras and, therefore, with a larger base. They show that the influence of the other parameters does not introduce greater error in the observation.

2.4. Processing Approach

The objective of this work is to establish an automatic system that allows us to detect premonitory movements in an area with several bad environmental conditions. We focus mainly on the existence of this displacement more than its magnitude.

Current and most commonly used photogrammetric systems allow us to create 3D point cloud models from several images automatically. We can establish the resolution of these models, but the point clouds generated with sets of images from two different times do not contain the same distribution of points. Therefore, interpolation is needed to compare multi-temporal point clouds. On the other hand, to monitor the movement of specific points, we have to measure in the photogrammetric model by selecting the point on the photograph.

Accuracy in the creation of the models is conditioned by the geometry of the camera system position regarding the slope, as explained in Section 2.3. The characteristics of the area, an example is the Castellfollit case, condition not only the cameras’ location but also the number of cameras that can be installed.

For these reasons, our approach is to work directly on individual images. Correlation methods on multi-temporal image sets allow us to detect movement. To automate the process, several programs have been implemented in Python and C++. As initially, these movements can be small, the analysis will be carried out for several time intervals. When working on the image, we calculate projected vectors for the displacement. Therefore, the final step consists in creating the 3D photogrammetric model to measure their magnitude and the estimated error by means of manual measurement at the control points.

2.5. Image Processing

To avoid the double image (high contrast and shades in the images) produced by the protective glass of the housing, the images were captured early in the morning, 3 images per camera, and late in the evening, between 3 and 4 images per camera according to the season. After the capture, they were stored in the Raspberry and sent to the server for processing.

Since the trigger and magnitude of these movements are unknown, and whether they are sudden movements or occur gradually over time, we created a protocol to compare images of different temporality. Each day’s images were compared with those of the previous day, a week ago, a fortnight ago, a month ago and two months ago. In this way, it was easier to detect small-magnitude movements.

Therefore, once the day’s acquisition was finished, the system identified the day and searched for the stored images of previous dates, and selected those of higher quality. In winter, these images corresponded to the afternoon or the last image of the morning because there were a high number of foggy days due to the geographic location of the study area and its proximity to a river. The analysis carried out by [40] shows that the influence of the type of image (black/white, RGB, etc.) was negligible in this process, and the result was much more sensitive to the correlation window used. If the window used is much bigger than the object that we want to track its displacement, it could not be detected. For this reason, we worked with the original RGB images directly without performing any kind of stacking or color space change.

An adjustment had to be made to correct possible camera movements before the time series comparison analysis. This process had to be performed once the images had been selected and for each of the cameras. After registering all the images for each camera, the next step was to compare the detection of possible movements. For this purpose, the cross-correlation method was used, Figure 4. If a movement or volume change was detected, the 3D model was created for quantification. This process was performed every day. Each of these phases is described in detail below.

A.

Image Adjustment

Although the cameras were on fixed supports, they were affected by small movements that could be due to the action of the wind or changes in temperature that could cause expansion and contraction of the supports or facades where they were installed. These movements were transmitted into small changes in the perspective of the images. Consequently, to compare the time series and detect variations, this change in perspective had to be corrected. To achieve this correction, an image was selected as a pattern, and the rest was resampled through a two-dimensional transformation, with translation as the main component. If the movements were very small, an affine transformation was enough to transform the coordinates of the images to be registered to the coordinates of the pattern image.

In the case of the study, due to changes in lighting and fog and after rain, which darkens the material, the scale-invariant feature transform (SIFT) method provided the best results. This method allows us to identify the image’s relevant features and match them among different images [41]. Due to the abundant, changing vegetation in the area, the search for significant points was carried out by applying a mask to avoid these areas and not consider the entire image.

Vegetation masks are usually used in the final 3D models that are obtained prior to the comparison of the 3D models [37]. However, in this case, the presence of a large number of plant masses on the wall that undergoes considerable changes over time makes their use essential in this first phase (A). To obtain these masks, there are various indices based on the visible RGB channels [42,43]. Among them, the Green Leaf Index (GLI) was selected as it offers better results in such an abundant, varied vegetation area. With the use of these masks, the calculation of the adjustment process was improved in computation time, which was reduced by half, and in the reduction of the RMSE (root mean square error) obtained.

Once the possible characteristic points had been identified, as a first step, those that were correct had to be filtered out using a brute-force matching algorithm [44]. Once this first set had been identified, an epipolarization filter had to be applied to guarantee the quality of the points to be used in the transformation.

B.

Movement detection

Once all the images for each camera had been aligned, a comparison was made to detect possible movements. To achieve this, the cross-connection method [45] was used to compute the matrix correlation, Equation (4). It was applied in both directions to avoid erroneous results due to the presence of residual vegetation, despite using masks. As mentioned before, this process was carried out for different temporary sequences of images with the aim of detecting movement without ambiguities.

$$\rho \left(j,k\right)=\frac{{{\displaystyle \sum }}_{x=1}^N{{\displaystyle \sum }}_{y=1}^M\left[\left(I_c\left(x,y\right)-{\overline{I}}_c\right)\left(I_p\left(x,y\right)-{\overline{I}}_p\right)\right]}{\sqrt{{{\displaystyle \sum }}_{x=1}^N{{\displaystyle \sum }}_{x=1}^N{\left(I_c\left(x,y\right)-{\overline{I}}_{cp}\right)}^2}\sqrt{{{\displaystyle \sum }}_{x=1}^N{{\displaystyle \sum }}_{x=1}^N{\left(I_p\left(x,y\right)-{\overline{I}}_p\right)}^2}}$$

(4)

where ρ is the correlation value, I_c and I_p are the radiometric levels of each area, ${\overline{I}}_c$ and ${\overline{I}}_p$ are the average of the radiometric level in the pattern and adjustment windows and (x, y) refers to each pixel of the matrix.

In this process, it is essential to use an adequate sampling window, the size of which is conditioned by the dimensions of change to be detected. In the study, the objective was to detect movement of the basaltic columns, which we can consider homogeneous throughout the scene, so the window was established according to its dimension at image scale. The window sizes were set at 31 × 31 and 61 × 61 pixels.

Based on the correlation, it was possible to determine the displacement vectors, which allow identification of the areas where there is movement of the basaltic columns, and to detect the areas where changes occurred, which in this case were equivalent to the fall of the block.

The forward–backward cross-correlation process allows us to detect erroneous vectors. Since they are obtained from image A to image B and reciprocally, the system can check the coherence in the result. In this way, only vectors with same direction are considered correct and kept as a result of the correlation process.

Figure 5 shows two cases. On the left is the image to be compared (Figure 5a,e). The pattern is located on the right (Figure 5b,f). For each of the sampling windows, the correlation matrices are created (Figure 5c,g) with a size of 250 × 250 pixels. In the three-dimensional representation of these matrices, there are several local maximums, but the highest indicates the maximum correlation. If this occurs in the center of the window, i.e., in pixel 125, in both the x and y axis, it indicates that there has been no movement (Figure 5d). If this maximum is displaced, it indicates that the point has undergone a movement. In Figure 5h, the maximum in the X coordinate is found in pixel 108 and not in the center of the matrix correlation; therefore, it is a signal of a block’s displacement.

In both cases, the value obtained in the correlation is analyzed. When the correlation value is high, but the point is not located in the same position, the displacement vector is drawn. When the correlation is high, but there is no offset, the vector becomes a point. It must be considered that the vectors are projected, so they allow us to notice that the movement is taking place, but not its 3D magnitude. Finally, if there is no correlation, this vector is not drawn. This happens in the border areas of the masks due to the growth of the vegetation. The lack of correlation is what helps us to identify areas where changes have occurred in the terrain, that is, where a block has fallen.

C.

Model building. Movement quantification

Once a movement has been detected in the previous process, as the vectors are projected, the photogrammetric process should be applied to quantify the movement in true magnitude since this process allows us to obtain terrain coordinates from image coordinates.

In the process of adjusting the photogrammetric block and obtaining the 3D model, photogrammetric systems (such as Metashape software) use automatic searches for characteristic points in the image. The result is a point cloud with a quasi-regular resolution. Only in the case of the measurements of the tie and control points and the points of interest measured manually (points on edges, corners of windows and doors, etc.) do we have points that do not belong to that point cloud.

As in the adjustment explained in Section 2.3, several images were used for each camera in order to have more redundancies. The problem with uncertainties remains, even increasing the number of photographs, because it depends on the captured geometry and the fact that we only have three points of view from the fixed cameras’ position.

Notably, for correct measurement of the magnitude of movements, the points belonging to the dense point cloud should not be used in the measurement since these points do not have to coincide with the control points. Measurements must be made on the photogrammetric model, which in these systems is equivalent to identifying and measuring the point on the images manually. In other words, the photographic coordinates are transformed into ground coordinates based on the estimated parameters.

The process consists of obtaining a model for each day and time in which movements are detected and georeferencing each of them from 22 control points, Figure 6. These points must be spatially distributed as homogeneously as possible. It must be considered that these are natural points since the instability of the study area prevents the placement of targets. Some control points, from 3 to 5, have also been used to estimate the quality of the photogrammetric adjustment. All these points have coordinates given by topographical systems with a quality of 1–2 cm.

For the adjustment, a precision of 2 cm (planimetry and depth) was considered for the tie and control points to obtain a similar magnitude error in the adjustment (RMSE) for the tie points. The maximum value of the reprojection error in the image was 1 pixel. In the case of control points, the values obtained were slightly higher than those found at the tie, of the order of 1.5 to 2 times in the worst case. The reprojection error was maintained at 1 pixel as the maximum value.

Notably, the largest residual errors found in the models obtained were always in the basaltic areas at the edges of the model. This was probably due to lower quality in the topographical and photographic measurements and because they belong to the perimeter of the adjusted photographic block.

3. Results and Discussion

One of the main factors to be considered in the analysis of the results is the fact that this was not a laboratory test. In a real case, the characteristics of the location of the rock wall to be monitored condition the quality of the results. In Castellfollit, there is a north-facing wall, next to a river, covered with vegetation. Most importantly, the topography of the area conditions the geometry of the photographic capture for the generation of 3D models.

The results shown correspond to the time series from 16 September 2021 to 2 October 2022. During this period, some of the cameras had to be readjusted or replaced due to mechanical failure, so there may be gaps and offsets in the results, which will be discussed in detail.

Taking as a pattern the image of 15 September 2021, the movements of the cameras in their X and Z component were analyzed. The composition of both is shown in Figure 7 for this period. In this analysis, we had not considered the position changes in cameras C2 and C3 produced by manual manipulation when they were repositioned and readjusted, respectively. We found a maximum displacement value of approximately 25 pixels for C1, 18 pixels for C2 and 100 pixels for C3. Notably, while for C1 and C2, the X and Z components show similar behavior, this was not the case for C3, for which the Z component presented movement twice as high as the X component. We assume that this was due to the conditions in which the cameras were located. While C1 and C2 were on the facade of a building, C2 was more protected. C3 was on a wooden pole and, therefore, more exposed to wind changes and vibrations due to traffic or other reasons. After the offset produced in February, the trend and displacement magnitudes remain in the same range. It was not possible to establish a direct correlation with wind gusts because the meteorological station from which data are available is 10 km away, nor with the temperature, since no seasonal trend was observed.

As a result of the cross-correlation process, we obtained images with the disparity between the images taken in different periods (daily, weekly, monthly, etc.) for the three cameras. This allowed us to detect the movement of several columns located in the lower central zone of the study area. One of the objectives of the study was to detect whether precursor movements occurred before the fall. If the answer is yes, we must ask another question related to the frequency of these movements. The tendency is to think that the movement occurs just before fracture and fall, or if it occurs before, it is at certain moments without continuity. In Castellfollit, it was verified that for the lithology of the area and the existing formations, this movement constantly occurred during the monitoring time, Figure 7 left.

Figure 8 left shows the result of the cross-correlation vectors after applying the forward-backward process. In other words, the complete process (phases A to B) was carried out from the pattern to each image in the set and vice-versa. Therefore, the displacement vectors were more clearly reflected than when the process was only carried out in one direction, i.e., from pattern to daily image.

In this case, we establish a threshold to consider that there is a movement in the pixel in 0.6. Since we compare the image from several days at different hours, they are affected by changes in lighting and color. Before fixing this value for another area, it will be necessary to test because it is conditioned to environmental factors.

In Figure 8, the initial positions of the columns have been marked in red and blue. The columns have been displaced in the image due to the toppling process. This process is reflected in the vectors drawn in Figure 7. The columns finally fell in May and August, respectively.

Figure 9 shows the position of three points that are considered fixed, F1, F2 and F3, and two that could move, called M1 and M2. Measuring in the image of camera 2 the vectors from September 2021 to September 2022, the projected movement produced in this period could be determined. The graph in Figure 9 shows that fixed points F1 and F3 suffer a movement of less than one pixel while F2 would reach two pixels, that is, the measurement noise. In contrast, it can be seen that points M1, M2 and M3 suffer continuous displacement, reaching 17.5 pixels before the fall of the column marked in red in Figure 8, 2.5 pixels for the column marked in blue and finally 6.5 pixels for the column located behind the one marked in red, corresponding to the point M3.

As we have mentioned previously, working with images allows us to observe if a movement has happened and the projected vector of this displacement. In order to know its true magnitude, we need to take it to a 3D coordinate system, i.e., transform the image coordinates to terrain coordinates through the photogrammetric process.

In the photogrammetric model, not on the point cloud, we have identified the same points F1, F2, F3, M1, M2 and M3, and measured the true magnitude of their final displacements before the fall of the column.

Table 2. Differential of the X, Y and Z coordinates and total displacement of the measured points. Units in meters.

	X (m)	Y (m)	Z (m)	D (m)
F1	0.008	0.050	0.025	0.040
F2	0.006	0.026	0.007	0.028
F3	0.015	0.025	0.017	0.033
M1	−0.275	−0.262	−0.065	0.385
M2	−0.106	−0.217	−0.088	0.257
M3	−.079	−0.230	−0.301	0.386

shows the true magnitude of these movements, whose main component is the depth at all points, between 0.2 and 0.27 m, reaching a displacement of approximately 0.4 m before the fall at point M2 in May 2022. The standard deviation of the three coordinates component D, according to the errors estimated in Section 2.3 for the worst conditions, and applying the covariance low, is 0.052 m. We also found a change in elevation, coordinate Z, for the point M3. This was due to the rotation of the column.

The uncertainty in the measurement in the photogrammetric models is higher than in the image whose standard deviation average in the three fixed points is 1.5 pixels, which corresponds to approximately 1 cm at the image scale.

These points were measured on the photogrammetric model by identifying them on the images. Thus, the identification was better than on the point cloud but not as accurate as in the case described for the targets, Section 2.3, since the columns have slightly rounded edges. A change of illumination can also influence the identification of a point.

The uncertainty in the measurement in the photogrammetric models is higher than in the image whose standard deviation average in the three fixed points is 1.5 pixels, which corresponds to approximately 1 cm at the image scale.

The models were generated considering only the area that appears in the images of the three cameras. Therefore, we avoid errors in the edges of the model due to a lower redundancy of observables. In addition, these edge zones, on the top and bottom part of the model, correspond to vegetation and buildings that are not of interest to our analysis.

During the photogrammetric process, the geometric distortion for the lenses of the three cameras was calculated and corrected. The residuals of the corrected images are less than 1 pixel.

In the case of working directly with the images by cross-correlation, as this geometric aberration remains constant over time, it will not influence the comparison of the images in time series.

Another result of the cross-correlation process is the information on the areas where changes have occurred in the monitored wall. Pixels with a lower correlation indicate areas where a change has occurred. In Figure 10, these points are marked in red, while pixels where no change has occurred are in blue. When a big change is produced between two different epochs, for example, when a block has fallen, the value of correlation between the pattern window and the image is very low. It also happens when the vegetation grows and overtakes the masks. Therefore, most of the low correlation pixels are located at the boundaries of vegetation areas, as they are not covered by the masks, except in the case marked with the red rectangle, where there is a higher concentration of pixels corresponding to the fallen column. The missing blocks could be detected because the concentration of points with low correlation is higher than in the case of vegetation growth. This leads us to consider that, in the case of wanting to process the image later in an automated way, it would be necessary to create an “influence” area around the masks. That is, a buffer should be created that allows us to consider these changes due to vegetation movements or growth. Another option is to create a mask for each image and not for periods, as implemented, although this entails a longer processing time.

4. Conclusions

The aim of this study was to analyze the validity of a continuous monitoring system for premonitory rockfall detection based on the implementation of time-lapse fixed position systems with three DSLR cameras, using currently available open-source hardware and software for data capture. A workflow was developed in which, based on the time series of images obtained from each of the cameras, ground movements, in this case of the basaltic columns, can be detected using cross-correlation in a forward-backward process. Once detected, since it is a stereoscopic system, we can achieve 3D photogrammetric models to estimate the real movement.

Working on a real case, the geometry of the image acquisition conditions obtaining the 3D models. In Castellfollit, the base–object ratio is within the advisable range between 5 and 10. Carrying out a test, we verified that the real errors using targets or artificial elements are within the theoretical estimated range due to the geometry and characteristics of the camera and despite the conditioning factors that are independent of them. As expected, the results that are obtained are slightly worse when working on natural points. For them, the unambiguous identification of points in the 3D model becomes more difficult since working on natural surfaces and not on pre-specified targets can lead to small errors, even if we work on the photogrammetric model and not on the point cloud.

Other limitations are constraints due to the location. These are insurmountable when a certain area has to be monitored. In the study case, the constraints were the north orientation of the rock face, with the sun’s rays falling on the cameras for several hours, which made it infeasible to use the images obtained in those time slots. In addition, fog due to the river made some of the images useless, so the capture rate could not be lowered if daily monitoring was desired.

However, the cross-correlation methods in non-regular time intervals allowed precursor movements of rockfalls of magnitude around a centimeter to be detected, which was the main objective. This leads us to consider that, at least for this type of formation, the system that was implemented can be used as a monitoring system for more detailed control.

Moreover, this correlation method allows us to detect major changes on the images. In this way, fallen blocks can be identified. The area where a block is missing is highlighted by clusters of uncorrelated pixels, and we have to pay attention to false positives due to the vegetation growing out of the mask.

In the comparison process, special attention must be paid to the correction of the images due to the change of perspective caused by the natural movement of the cameras. In addition, the use of masks to remove vegetation in the correlation process is essential, with a double objective: to reduce the computation time in uninteresting areas and to avoid erroneous data.