Visualization of Concrete Slump Flow Using the Kinect Sensor

Abstract

Workability is regarded as one of the important parameters of high-performance concrete and monitoring it is essential in concrete quality management at construction sites. The conventional workability test methods are basically based on length and time measured by a ruler and a stopwatch and, as such, inevitably involves human error. In this paper, we propose a 4D slump test method based on digital measurement and data processing as a novel concrete workability test. After acquiring the dynamically changing 3D surface of fresh concrete using a 3D depth sensor during the slump flow test, the stream images are processed with the proposed 4D slump processing algorithm and the results are compressed into a single 4D slump image. This image basically represents the dynamically spreading cross-section of fresh concrete along the time axis. From the 4D slump image, it is possible to determine the slump flow diameter, slump flow time, and slump height at any location simultaneously. The proposed 4D slump test will be able to activate research related to concrete flow simulation and concrete rheology by providing spatiotemporal measurement data of concrete flow.

1. Introduction

Monitoring the workability of freshly mixed concrete is necessary to ensure that the concrete is properly placed during construction and that adequately hardened strength is achieved after construction [1]. Both are related to structural integrity, safety, and construction productivity. For the safe and easy construction of high-quality concrete structures, it is necessary to use fresh concrete with proper workability. Workability such as flowability, viscosity, yield stress, and resistance to material separation required at construction sites depends on the type of target structure, the spacing of reinforcements, dimensions, cross-sectional shape, construction method, and concrete pumping distance [2]. Monitoring of concrete workability [3,4], as well as strength [5,6,7], is regarded as a key technology in the long-distance transport of concrete required for high-rise buildings and long-span bridge construction. In the field of innovative concrete 3D printing technology in the construction area, workability control will be critical to ensure consistent material properties and structural performance [8,9,10].

In the field of concrete materials, highly flowable self-consolidating concrete (SCC) is attracting attention for its ability to fill congested rebar spaces in a formwork under its own weight without using a vibrator. The characteristics of SCC are filling ability, passing ability, and stability [11]. The use of SCC can increase the construction efficiency because it can improve the quality of a concrete structure, reduce the labor requirement, and increase the construction speed. The use of SCC is expected to increase as the complexity of building design and the number of high-rise buildings increase [4].

As a standard test method to assess the filling ability of flowable concrete, the slump flow test shown in Figure 1 has been established [12,13,14]. It is the simplest and most widely used test method for SCC [1]. The result of the slump flow test is recorded by measuring the diameter of the circular spread of the concrete after lifting up the slump cone. As non-mandatory information, relative viscosity can be measured during the slump flow test [13]. The T50 value indicating the relative viscosity is the time for the outer edge of the spreading concrete mass to reach a diameter of 500 mm. Most test methods, such as the slump flow test and the slump test [15], are carried out by operators using a ruler or a stopwatch, and hence measurement errors are inherent and the measured value may vary depending on the operator. There is also a possibility that the obtained value can be wrongly recorded or easily manipulated after measurement. If concrete with insufficient workability is used for the construction of structures because of inaccurate measurement results or incorrect data records, there will be future problems in terms of structural safety.

With the development of SCC and high-performance concrete, instead of the conventional test devices, a rheometer has been introduced to measure the rheological parameters of concrete [16]. For example, it has been used for workability monitoring of high-rise buildings [3,4]. Recently, concrete flow based on a computational fluid dynamics (CFD) analysis has been actively studied with a rheometer [17,18]. However, there is a lack of understanding about concrete rheology in the construction field. Furthermore, with the price of a rheometer ranging from $20,000 to $180,000, its use in the field is limited by its costliness. The workability measuring equipment required in the construction field thus should be able to quantitatively evaluate the workability at low cost.

In this study, a new framework for qualitatively evaluating concrete workability using a depth sensor is proposed. Kinect, a low-cost depth sensor, is utilized to measure the dynamically changing 3D concrete surface during a slump flow test, as shown in Figure 2. The Kinect was originally developed for human motion recognition in the X-box, a game console developed by Microsoft Corporation [19]. Its resolution and its scanning speed are excellent and due to mass production it is inexpensive. Its use is expanding in various fields, such as robotics [20,21], rehabilitation engineering [22], and even in the construction field in monitoring structural health [23,24]. Visualization of spatiotemporal data using sensors is useful for numerical analysis. Just as the infrared thermography is useful for heat transfer analysis of concrete surfaces [25], the 4D slump test using Kinect will activate research on concrete flow simulation.

This paper is organized as follows. Section 2 presents the data processing algorithm, related to the workability test, which can produce the spreading diameter over time and slump flow time from the Kinect data. The data acquired in the depth sensor are represented in the coordinate frame on the ground plane by coordinate transformation, which is well used in computer graphics and robot kinematics. The transformed point cloud data are then used to create the surface of the concrete and the surface is reorganized in a grid form for further data processing. A final 4D slump image is constructed by collecting the extracted cross-sections of the concrete slump at each instant surface image. In Section 3, the experimental setup for 4D slump, including the test material and procedure, is described. In Section 4, the experimental results show that concrete flow visualization is possible during the slump flow test by processing the time-varying surface shape of the concrete.

2. Data Processing Algorithm for the 4D Slump Test

The Kinect is used to acquire spatial information of the dynamically changing surface of fresh concrete during the concrete slump flow test. The raw data cannot be directly utilized to analyze useful information on the concrete workability; several stages of data processing are needed to produce slump flow diameter, slump flow time, and slump height. The proposed algorithm for data processing consists of the following procedures: (1) representation of 3D spatial information at the camera frame {C} from the depth image; (2) determination of a ground plane equation; (3) calculation of a transformation matrix between the camera frame {C} and the slump frame {S} where the slump cone is initially located; (4) reconstruction of the 3D surface for concrete slump; and (5) cross-section extraction and construction of a 4D slump image. Each procedure is described in the following sections.

2.1. Representation of 3D Spatial Information at Camera Frame {C}

While projecting a known speckle pattern of near-infrared light, the Kinect sensor acquires disparity images at the IR camera in real time [26], where the disparity d is represented at the pixel location (u, v). Figure 3 represents the pinhole camera model, which shows how the point in three-dimensional space is projected onto the image plane of the IR camera. The point P(X,Y,Z) described in the camera coordinates is projected to the point p(u,v) on the image plane, which is apart from the camera frame by a focal length f. Here, the coordinate system of (X,Y,Z) is referenced to the camera frame {C}. Using the proportionality of two similar triangles OPQ and Opc, the following equations for coordinates X and Y are obtained:

$$\begin{gathered} X=\frac{\left(u-c_x\right)Z}{f} \\ Y=\frac{\left(v-c_y\right)Z}{f} \end{gathered}$$

(1)

where f is focal length and c_x and c_y are optical center pixels in the IR camera. The focal length and optical center pixels can be determined by a standard calibration of the camera.

Depth Z is expressed as a nonlinear function of disparity d [21]. The mathematical model between depth and disparity d is derived from the geometric relationship in Figure 3, which is given by Equation (2) [27]:

$$Z=\frac{Z_0}{1+\frac{Z_0}{fb}d}$$

(2)

where Z₀ is the distance to the reference plane, b is the baseline between the IR projector and camera. Since the output value by the sensor is the normalized disparity d’ in practice, which ranges from 0 to 2¹¹ − 1, d should be replaced with md’ + n [27,28] and then Equation (2) becomes as follows:

$$\frac{1}{Z}=\left(Z_0^{-1}+\frac{n}{fb}\right)+\left(\frac{m}{fb}\right)d^{\prime }=C_0+C_1{d}^{\prime }=\left[\begin{array}{cc}1& d\prime \end{array}\right]\left[\begin{array}{c}{C}_0\\ C_1\end{array}\right]$$

(3)

Equation (3) includes five parameters such as Z₀, b, f, m, and n but depth Z can be determined by two parameters, C₀ and C₁. These parameters should be estimated by depth calibration. In the calibration experiment, several sets of disparity information and the measured depth are collected. Since the inverse of depth has a linear relationship with the disparity in Equation (3), the calibration parameters C₀ and C₁ can be calculated in terms of pseudo-inverse form by the least square method. By calibrating the parameters, the systematic error due to Z₀, b, and f can be eliminated and depth accuracy can be improved [28].

To sum up, the final coordinates of X, Y, Z for the given (u,v,d’) are calculated as follows:

$$\begin{array}{ccc}X=\frac{\left(u-c_x\right)Z}{f},& Y=\frac{\left(v-c_y\right)Z}{f},& Z=\end{array}\frac{1}{C_0+C_{1}d\prime }$$

(4)

Using the above equations with the calibration parameters, it is possible to reconstruct the points in 3D coordinates again from the disparity image.

2.2. Determination of the Ground Plane Equation

During the slump flow test, the surface of the concrete is detected by a Kinect depth sensor, as shown in Figure 4. There are two coordinate systems, {C} and {S}. {C} is the camera frame where the Kinect IR camera is located and {S} is the slump frame where the concrete slump is located. In order to represent the points P(X,Y,Z) in the camera frame {C}, the expression of ^CP is used in this paper. Here, the leading superscript indicates the coordinate system to which the points P are referenced. For the data processing of the measured point cloud, it is convenient to use a description of the points in frame {S}, that is, ^SP. In order to change the description from frame {C} to {S}, the coordinate transformation can be adopted as a mathematical tool. The theory of coordinate transformation is well established in the field of robot kinematics to express the position and orientation of a robot’s end effector, which is serially connected by several links and joint angles [29]. As the first step, the slump frame {S} should be mathematically represented in the camera frame {C}.

In Figure 4, the slump frame {S} is defined as a frame whose xy plane is the ground plane and whose Z-axis is the opposite direction of gravity. In order to transform ^CP to ^SP, the relative rotation matrix ${}_S{}^{C}R$ and translation vector ^SP_SORG are required. In order to calculate the rotation matrix ${}_S{}^{C}R$, it is basically necessary to obtain the plane equation of the ground surface where the slump test is performed.

In this paper, the RANSAC (RANdom SAmple Consensus) algorithm [30] is utilized to obtain the ground plane equation. This algorithm is an iterative method to estimate parameters of a mathematical model from a set of observed data containing outliers. Here, it is used to estimate the parameters (a, b, c, and d) of the following equation from a set of selected data on the ground plane around the slump.

$$a\mathit{X}+b\mathit{Y}+c\mathit{Z}+d=0$$

(5)

The unit vector of the Z-axis in frame {S} is the orthonormal vector of the ground plane. It is defined as v in this paper and is expressed as the parameters of the ground plane equation as follows:

$$v=v_x\widehat{i}+v_y\widehat{j}+v_z\widehat{k}=\frac{a}{\sqrt{a^2+b^2+c^2}}\widehat{i}+\frac{b}{\sqrt{a^2+b^2+c^2}}\widehat{j}+\frac{c}{\sqrt{a^2+b^2+c^2}}\widehat{k}$$

(6)

It is reasonable to use the RANSAC algorithm since it detects outlier data of Kinect and does not use them in fitting the plane equation. That is, unlike the least square method, it is possible to remove the effects of the extreme values resulting from erroneous measurement or environmental conditions such as light intensity or reflectivity. Figure 5a shows the depth image captured in the Kinect during the slump test. If four rectangular areas around the concrete slump are selected, the ground plane is generated by the RANSAC algorithm using the ground points at four rectangular areas, as shown in Figure 5b. The origin point of {S} can be selected as the center of four rectangular areas. The unit vector on the Z-axis of the frame {S}, v is used to calculate the relative orientation between the camera frame {C} and the slump frame {S} in the next section.

2.3. Coordinate Transformation from Camera Frame to Slump Frame

In order to describe the measured points of fresh concrete in slump frame {S}, the transformation matrix between {C} and {S} should be obtained. The relationship between {S} and {C} is characterized by the rotation matrix, ${}_S{}^{C}R$, as well as the translational vector, ^CP_SORG. ^CP_SORG is the vector from the camera frame {C} to the slump frame {S}.

When an orthonormal vector to the ground plane is represented as $\widehat{v}=v_x\widehat{i}+v_y\widehat{j}+v_z\widehat{k}$ in {C}, it is possible to find the rotation matrix of {C} relative to {S} based on the Z-Y-X Euler angles [29]. Any orientation can be achieved by three rotations about the axes of a moving frame in Euler angles. Figure 6 shows the axes of {S} after consecutive Euler angle rotations are applied. When the origin of frame {S} is initially coincident with that of frame {C} in Figure 6, rotation α about ${\widehat{Z}}_c$ causes ${\widehat{X}}_c$ to rotate into ${\widehat{X\prime }}_s$, and ${\widehat{Y}}_c$ to rotate into ${\widehat{Y\prime }}_s$. Here, the angle α is determined when the vector ${\widehat{X\prime }}_s$ is located in the plane formed by ${\widehat{Z}}_c$ and $\widehat{v}$. The next rotation is performed about an axis of the intermediate moving frame {S’}. It is possible to find β when rotation β about ${\widehat{Y\prime }}_s$ causes $\widehat{Z}{\prime }_s$ to rotate into ${\widehat{Z}}_s$, that is, $\widehat{v}$ (orthonormal vector to the ground plane). As a result of two consecutive rotations about the axes of the moving frame, the final orientation of {S} is given relative to {C} as:

$$\begin{gathered} {}_S{}^{C}R={}_S{}^{C}R{}_{Z^{\prime }{Y}^{\prime }{X}^{\prime }}(\alpha ,\beta ,0)=R_Z\left(\alpha \right)R_Y\left(\beta \right)R_X\left(0\right) \\ =\left[\begin{array}{ccc}c\alpha & -s\alpha & 0\\ s\alpha & c\alpha & 0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{ccc}c\beta & 0& s\beta \\ 0& 1& 0\\ -s\beta & 0& c\beta \end{array}\right]\left[\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& 1\end{array}\right]=\left[\begin{array}{ccc}c\alpha c\beta & -s\alpha & c\alpha s\beta \\ s\alpha s\beta & c\alpha & s\alpha s\beta \\ -s\beta & 0& c\beta \end{array}\right] \end{gathered}$$

(7)

The angles α and β can be easily expressed in terms of the components of the orthonormal vector v in frame {C}, as illustrated in Figure 7. α represents the angle between $v_x\widehat{i}+v_y\widehat{j}$ and the X-axis, and β refers to the angle between $v_x\widehat{i}+v_y\widehat{j}+v_z\widehat{k}$ and the Z-axis. They can be expressed as follows:

$$\begin{gathered} \alpha =ta{n}^{-1}\left(v_y,v_x\right) \\ \beta =acos\left(v_z\right) \end{gathered}$$

(8)

The homogeneous transformation matrix ${}_S{}^{C}T$ that maps ^SP to ^CP is represented by the orientation and position information as follows [29]:

$${}_S{}^{C}T=\left[\begin{array}{ccc}& {}_S{}^{C}R& \\ & & \\ 0& 0& 0\end{array}\begin{array}{c}{}_{}{}^{C}P{}_{SORG}\\ \\ 1\end{array}\right]$$

(9)

The columns of ${}_S{}^{C}R$ are unit vectors defining the directions of the principal axes of {S} and ^CP_SORG represents the position vector of the origin of {S} in frame {C}. The description of points with respect to the slump frame {S} is calculated by the inverse of ${}_S{}^{C}T$ as follows:

$$\begin{gathered} {}_{}{}^{S}P={}_C{}^{S}T{}_{}{}^{C}P \\ \mathrm{where}{}_C{}^{S}T=\left[\begin{array}{ccc}& {\left({}_S{}^{C}R\right)}^T& \\ & & \\ 0& 0& 0\end{array}\begin{array}{c}-{\left({}_S{}^{C}R\right)}^T{}_{}{}^{C}P{}_{SORG}\\ \\ 1\end{array}\right] \end{gathered}$$

(10)

Figure 8 shows the result of the transformation of points in the camera frame to the slump frame using Equation (10).

2.4. Reconstruction of 3D Surface for Concrete Slump

Triangulated irregular network (TIN) is the digital data structure generally used in geographic information systems (GIS) to represent the topographical surface. TIN is composed of contiguous, non-overlapping triangles. It can be used to convert the measured scattered data points into a model of the 3D surface. The surface of slump flow generated by TIN is shown in Figure 9a. Large triangles on the left part of Figure 9a are the shadow areas where shooting is restricted by the angle of the Kinect sensor. The vertexes of TIN are located at irregularly spaced points (x_i, y_i) in the plane, that is, TIN itself does not have equal intervals in the X- and Y-axes. For ease of data analysis, such as cross-section extraction, it is desirable to construct a network composed of regular grids, as shown in Figure 9b. Triangulation-based linear interpolation is applied to generate data at the grid points [31]. Each triangle plane is defined by three points in the equation z = ax + by + c, and any point located at a specific grid (x,y) within this triangle can be linearly interpolated. In this grid coordinate, it is easy to extract the slump height at a specific location of the ground plane.

2.5. Cross-Section Extraction and Construction of a 4D Slump Image

Through the above-mentioned data processing algorithm, the shape information obtained from the depth sensor camera can be expressed in a 3D surface where the height data are available at any grid position of the slump frame {S}. The dynamically changing shapes of the concrete slump over time are the four-dimensional spatiotemporal data, as shown in Figure 10. Here, every image of the dynamically changing slump flow shape is the result of applying the series of data processing procedures described above.

By selectively extracting the information from each slump in Figure 10, it is possible to reconstruct the data and visualize the workability effectively. For example, the cross-sectional curve (2D) of a slump can be collected over time (1D) and reconstructed into three-dimensional data, which again can be compressed into a 2D image, as seen in the top view. In this paper, the compressed 2D image is called a 4D slump image and it will be shown in Section 4 with an explanation of the experiment in Section 3.

3. Experimental Setup for the 4D Slump Test

3.1. Devices for the 4D Slump Test

As a low-cost 3D depth sensor, Microsoft Kinect was utilized to acquire data during the slump test. Kinect consists of an IR laser pattern projector, an IR camera, and an RGB camera. When the IR projector projects a laser speckle pattern onto an object, an image of the pattern on the object is captured by the IR camera and then processed to reconstruct a 3D map of the object [26]. The Kinect provides a depth video stream with a resolution of 640 × 480 pixels at a maximum of 30 frames per second. At a lower frame rate, it can deliver 1280 × 1024 pixels. The default sensing range is 0.8 m to 6 m but it is configured to near mode, which provides a range of 0.4 m to 3 m for a better quality of resolution [32]. Depth resolution refers to the minimum detectable difference in a certain continuous distance range [33]. Based on research by Smisek et al., the resolution is 0.65 mm at 0.5 m and it changes with the distance in a quadratic manner [19]. The depth resolution is less than 2 mm at 1 m and 25 mm at a 3-m distance [27]. According to the investigation of Khoshelham et al. on the Kinect, the random error of the depth measurement also increases in a quadratic manner as the distance of the object increases [27]. The random error is less than 0.6 cm at a distance of 1 m but it reaches 4 cm at the maximum range of 5 m. During the experiment, the distance was kept to less than 1 m where Kinect had a resolution of 2 mm and a random error of 6 mm. It is not recommended to increase the distance greater than 1 m because the quality of the depth data is degraded by random error and resolution.

The Kinect was assembled with an in-house mounting frame, as shown in Figure 11a. The incorporation of several bolt holes on the mounting frame enabled a better shooting angle for measurement by providing additional freedom to connect it to the camera tripod. The coordinate transform algorithm allowed shooting from any angle with Kinect. Kinect drivers such as OpenNI and NiTE were installed on a Windows 7 laptop computer and an in-house MATLAB code was run for measurement and analysis.

3.2. Test Material and Procedure

As a flowable concrete, an SSC mixture with a water-to-binder ratio of 39% was used in the experiment. The proportions of concrete mix are shown in

Table 1. Mixture mass proportions of test material used in the experiment.

W/B	Water	C	FA	S	G	SP
39%	9.37%	16.77%	7.18%	35.01%	31.52%	0.345%

C: Ordinary Portland cement, ρ = 3.16, s = 3214 cm²/g; FA: Flay ash, ρ = 2.19, s = 3960 cm²/g; S: Silica Sand, ρ = 2.65, diameter from 0.1 mm to 0.6 mm; G: Gravel, diameter less than 13 mm; SP: Polycarbonate Superplasticizer.

. The diameter of the silica sand used in the mix ranged from 0.1 mm to 0.6 mm. The diameter of the coarse aggregate was less than 13 mm.

The slump flow test was performed based on ASTM C1611 [13]. The cone was filled with fresh concrete to the struck volume in the inverted position on a flat, level, non-absorbent surface, as shown in Figure 11b. When the cone was lifted, the concrete flowed out. The slump flow test was recorded in 24 frames per second by a laptop connected to the Kinect sensor while the raw depth data were displayed on the monitor. In ASTM C1611, slump flow and T50 are measured by operators. Slump flow is the average of two perpendicular diameters across the spread of concrete and T50 is the time for the fresh concrete to reach a diameter of 500 mm after lifting the slump cone. In the conventional method, it is hard to start and stop a clock at exactly the correct times while conducting the slump flow test. The slump flow time (T50) in the inverted slump cone orientation generally gives a higher value than in the normal orientation [34,35].

4. Experimental Results

The depth images recorded at 24 Hz sampling frequency during the slump flow test were processed by the 4D slump processing algorithm described in Section 2.1, Section 2.2, Section 2.3 and Section 2.4. As a result, reconstructed images of the concrete surface in regularly spaced coordinates of the slump frame at every time frame were obtained, as shown in Figure 12. It was then possible to visualize the spreading cross-section of fresh concrete during the slump flow test, as shown in Figure 13. This 3D surface is made of the extracted 2D cross-sections in a given 3D concrete surface at a specific time frame of Figure 12. By representing the height z at a specific location (x, y) of Figure 13 as color information, the 4D slump image is obtained, as presented in Figure 14. This is a compressed image of the slump flow test showing the dynamically spreading diameter of concrete slump flow. It contains the generalized slump flow time Ti, the slump diameter at any time, and the height of slump at any time and location. In Figure 14, the local area in red at the initial time frame is the result of the concrete drop from the slump cone.

In the 4D slump image, it is possible to detect the boundary of the slump flow, as presented in Figure 15. By utilizing the characteristics of the 4D slump image where the slump flow diameter increases from the left to right side, the boundaries of the slump flow were searched along the vertical axis from the upper and lower border lines to the middle. For an edge detection algorithm, the simple threshold method or Sobel operator could be applied. In Figure 15, the red line represents the moving average of the detected edge points plotted with blue color. After detecting the upper and lower boundaries of the slump flow in the 4D slump image, the slump flow diameter over time was obtained, as shown in Figure 16, by calculating the relative distance between the two boundaries in Figure 15. This graph provides useful information for comparing the experimental behavior of concrete flow with simulation results through a CFD analysis.

5. Conclusions

In this paper, a novel workability test method using a 3D depth sensor was proposed and the dynamically spreading diameter of concrete slump flow was successfully visualized through a 4D slump processing algorithm. In order to represent the surface of fresh concrete in the slump frame, we transformed the depth data in the camera frame {C} acquired by Kinect into the data in slump frame {S} with the calculation of the relative orientation and position between the {C} and {S} frames. The scattered data in the slump frame were then interpolated in the regularly spaced grid and the cross-sections of the slump flow at each time frame were collected. Through the experiment, it was confirmed that the information on the spreading slump flow diameter, slump height, and slump flow time could be obtained by a single 4D slump image using the proposed 4D slump processing algorithm. In the conventional concrete workability test device, it is impossible to obtain these kinds of images and graphs.

The proposed 4D slump test method offers several advantages. First, instead of human measurements using a ruler and a stopwatch, which can introduce human error, it can digitize the workability data and quantitatively evaluate the workability at low cost. Second, it affords non-contact, portable digital measurement and provides additional information besides the basic information while performing the existing workability test. Third, measurement of concrete flow will be helpful in the quality monitoring of concrete by providing a digital record that cannot be easily modified. In addition, it will accelerate the development of flowable concrete based on numerical simulations by providing digital data for concrete flow experiments. Beside systematic error, random error, and resolution discussed in this paper, the depth measurement quality is influenced by the light condition and target reflectivity. At a high light intensity, the IR camera cannot detect laser speckles on the object since the relative contrast of the speckles is decreased. That is to say, the signal-to-noise ratio of the reflected pattern is low under sunlight. If the water content of fresh concrete is high or the water in the fresh concrete tends to rise to the surface, speckle patterns projected by an IR projector are not reflected back from the concrete surface. Consequently, the Kinect will create gaps and outliers [27,36]. In terms of its usability, it can be used for a slump test, the most popular workability test, since this test only requires measurement of the static height of the concrete slump.