Next Article in Journal
EWOA-OPF: Effective Whale Optimization Algorithm to Solve Optimal Power Flow Problem
Previous Article in Journal
Attack and Defense Strategy of Distribution Network Cyber-Physical System Considering EV Source-Charge Bidirectionality
Previous Article in Special Issue
Artificial Jellyfish Search Algorithm-Based Selective Harmonic Elimination in a Cascaded H-Bridge Multilevel Inverter
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Electronics
Volume 10
Issue 23
10.3390/electronics10232974
electronics-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Unmanned Aerial Vehicles for Crowd Monitoring and Analysis

by
Muhammad Afif Husman
1,*,
Waleed Albattah
2,
Zulkifli Zainal Abidin
1,
Yasir Mohd. Mustafah
1,
Kushsairy Kadir
3,
Shabana Habib
2,
Muhammad Islam
4 and
Sheroz Khan
4,*
1
Department of Mechatronics, International Islamic University Malaysia, Kuala Lumpur 53100, Malaysia
2
Department of Information Technology, College of Computer, Qassim University, Buraydah 52571, Saudi Arabia
3
Electrical Section, Universiti Kuala Lumpur British Malaysian Institute, Gombak 53100, Malaysia
4
Department of Electrical Engineering, College of Engineering and Information Technology, Unaizah Colleges, Unaizah 2053, Saudi Arabia
*
Authors to whom correspondence should be addressed.
Electronics 2021, 10(23), 2974; https://doi.org/10.3390/electronics10232974
Submission received: 4 October 2021 / Revised: 11 November 2021 / Accepted: 23 November 2021 / Published: 29 November 2021
(This article belongs to the Special Issue Application of Internet of Things, Big Data and Artificial Intelligence in Electrical Energy and Power Systems)
Browse Figures
Versions Notes

Abstract

:
Crowd monitoring and analysis has become increasingly used for unmanned aerial vehicle applications. From preventing stampede in high concentration crowds to estimating crowd density and to surveilling crowd movements, crowd monitoring and analysis have long been employed in the past by authorities and regulatory bodies to tackle challenges posed by large crowds. Conventional methods of crowd analysis using static cameras are limited due to their low coverage area and non-flexible perspectives and features. Unmanned aerial vehicles have tremendously increased the quality of images obtained for crowd analysis reasons, relieving the relevant authorities of the venues’ inadequacies and of concerns of inaccessible locations and situation. This paper reviews existing literature sources regarding the use of aerial vehicles for crowd monitoring and analysis purposes. Vehicle specifications, onboard sensors, power management, and an analysis algorithm are critically reviewed and discussed. In addition, ethical and privacy issues surrounding the use of this technology are presented.

1. Introduction

Crowd monitoring and analysis have been becoming vital from public security and safety viewpoints because crowd participants may show abnormal behavior. An increase in crowd density and the associated abnormal behavior amongst its participants may ultimately lead to stampede incidents with risk of injuries. The probability of risks multiplies when coupled with strict spatiotemporal constraints, such as those exercised in religious gatherings [1,2]. Furthermore, in such mass gatherings, potential public health threats are even more severe, ranging from transmission of infectious diseases, thermal disorders, the possibility of terrorism incidents, and violent crowd behaviors resulting from alcohol consumption and/or substance abuse [3]. Planned religious gatherings can attract millions of people into designated areas enhancing societal values. For example, Hajj is considered one of the largest planned mass gatherings where over two million Muslims gather annually in Mecca [1]. Stampede incidents in Hajj and other mass religious gatherings, such as the Kumbh Mela in India, are partly attributed to abnormal crowd behaviors, resulting in panic and subsequently fatal accidents [1,2].
Traditional crowd analysis methods have relied on visual inputs obtained from static or fixed-location cameras that record images or videos, resulting in fixed angle visibility and limited coverage. In addition, fixed visual inputs cannot, and are hence unable to, perform persistent and continuous tracking of moving crowds, unless the deployment of some massive monitoring devices’ network is made in place. In recent years, unmanned aerial vehicles, commonly known as drones have been deployed to perform crowd analyses, complementing the fixed monitoring devices [4]. Drones, primarily used for military applications, are now gaining interest in their use to capture footage that would otherwise require the deployment of helicopters and manned aircrafts. Specifically, the use of drones provides the following advantages: (1) ability to be equipped with required sensors and payloads for acquisition of additional metrics other alongside visual data [5], (2) availability of real-time data for crowd dynamics modeling, with the help of powerful onboard processing units for estimating the crowd dynamics [6,7,8], and (3) lowering overall operational costs as the same monitoring device can be deployed elsewhere with an effective increase in its coverage [4], as well as reducing the human resource [5,9]. The benefits provided by drone use have already been established in other fields where mobility and aerial access tremendously increase visibility and site access such as in the maritime environment [10], agricultural technologies [11,12,13,14], mining industries [15], and disasters, tsunami, and pandemic management [16].
The crowd monitoring and analyses discussed in this paper can be further divided into several domains, namely crowd detection [8,17,18,19,20,21,22], crowd counting [8,23,24,25,26,27], crowd density estimation [17,26], crowd tracking [6,22,28,29], and crowd behavior analysis [30,31]. Whereas most works have focused on a single domain, recent results from the literature have discussed algorithms developed to tackle multiple domains such as the one presented in [32], and similar other areas will be further discussed in the proceeding literature.
To the best of the authors’ knowledge, a balanced review on crowd monitoring and analysis using drones, including a much-needed discussion on the ethical and privacy issues surrounding the application of drones in crowd analysis, is scarce. This paper attempts to gather relevant literature resources to address the technological aspects alongside the ethical aspects of such advancements.
The rest of the paper is organized as follows: Section 2 describes the drone architecture reported in the recent literature, consisting of onboard sensors and power management systems by including commercially developed solutions where relevant. Section 3 elaborates on the applications of drones in crowd management and the accompanying technologies used for monitoring and analysis of crowds. This includes emerging algorithms for crowd analysis, as well as recent advancements in crowd prediction. Section 4 presents privacy and ethical issues surrounding the use of drones in terms of their societal impact and the legislature and global acceptance of drone technology. Finally, the conclusions are drawn as a concise account in Section 5.

2. Drone Architecture

Multiple drone architectures exist in the literature, ranging from classification based on functions, classification by weight and size, performance characteristics, and engine types [33]. This section discusses the architecture of drones used for crowd monitoring and analyses, covering its build, onboard sensors, communication, and power management strategies.

2.1. Drone Build

Arguably, the vast selection of drone architectures is reduced when used for the monitoring of human crowds. A significant proportion of drones used in the literature point to multi-rotor drones with a vertical take-off and landing (VTOL) mechanism. The choice of multi-rotor VTOL drones is advantageous in several aspects. Firstly, VTOL is preferred to other mechanisms since it does not require additional launching platforms such as runaways or catapults [34,35]. This allows easy configuration and fast deployment for crowd monitoring purposes and requires a smaller deployment area. Secondly, multi-rotor VTOL drones can hover in one place, making them the preferred choice for monitoring as they can be positioned above the crowds [18]. This is especially useful for still crowd imaging, as well as continuous crowd monitoring applications. The second choice of architecture presented in the literature, albeit scarce, is the fixed-wing type drones, which have the advantage of longer flight endurance and higher efficiency.
Of the drones addressed in the literature, the majority chose off-the-shelf commercial drones for the said purpose, notably from the manufacturer DJI, which in the year 2020 accounts for more than 70% of the drones market share in the US [36]. The choice of off-the-shelf drones is likely since they come as an integrated package that includes drone build, flight controller, mission planning system and data transmission link, allowing for plug-and-play deployment. However, some work mentions utilizing a custom-made drone to allow full access to the underlying hardware requirement, especially to allow onboard image processing. Shao et al. [30] describe using a quadrotor with a NVIDIA Jetson TX1 embedded computer to carry out real-time image processing coupled with STM32F427VIT6 based flight controller. Custom hardware deployments allow complete control of the hardware interfacing and remove restrictions due to proprietary software and codes shipped with the commercial drones.

2.2. Visual and Onboard Sensors

No doubt, the presence of a visual sensor is a requirement for crowd-related drone activities. The RGB or visible light type camera is the most widely used, most commonly equipped with commercial drones [18,23,26,28,32,37]. On top of RGB, the addition of thermal camera is another visual sensor that is either used on its own [21,30] or used in combination with its RGB counterpart [18,23].
RGB images have been the gold standards for image processing applications. RGB images often have higher resolution and contain richer details regarding the area being surveyed [18]. However, RGB images are often affected by the environment [30], such as its susceptibility to illumination changes or quality degradation due to limited lighting (for example during nighttime captures) [23]. Thermal imagery complements these disadvantages by providing heat signature of the objects in the area of interest, regardless of the environmental illumination conditions. However, thermal images often suffer from lower spatial resolution [19].
Although commercial drones often come with additional onboard sensors such as the Global Positioning System (GPS), most of the crowd-related literature did not describe taking advantage from these sensors. Bhattarai et al. [37] have described the use of onboard GPS, in addition to the RGB camera, to geo-locate humans detected from aerial drone surveillance activity. This in turn, has allowed the authors to visualize the location of the detected human on a Geographic Information System (GIS) platform such as Google Earth. Similarly, Singh et al. [31] have discussed the use of the onboard inertial measurement unit (IMU) to calculate the drone’s horizontal velocity. In both cases, built-in or mounted onboard sensors were used as an augmentative feature to obtain additional data during the drone operation.

2.3. Communication

In a single-drone deployment scenario, the primary communication link is bi-directional between a drone and its Ground Control Station (GCS). Commercial drones such as DJI uses proprietary communication protocols such as Ocusync and Lightbridge, in addition to the WiFi transmission system. Open-source protocols also exist for building communication links for custom-made drones. One such protocol is the Micro Air Vehicle link (MAVLink), which employs a lightweight binary serialization protocol [38]. As the complexity of operation increases, communication can shift from being centralized GCS to a decentralized drone-to-drone communication [39].
Most works in the literature concerning aerial crowd monitoring lack discussion regarding the communication architecture, most likely suggesting the use of the built-in communication protocol bundled with the purchased drone. Several works in this domain, however, have discussed additional details in the communication methods. In [28], the authors have presented a framework for persistent crowd tracking dubbed PERCEIVE which consists of a swarm of drones carrying out video surveillance using a novel charging scheduling and mobile ground charging station. The authors cited improvement in efficiency in comparison with a fixed charging station or no charging condition. In a crowd analysis application presented in [31], the authors have described using cloud computing to analyze images on the cloud due to the memory-intensive computation required by the corresponding algorithms. This allows offloading a segment of intensive drone tasks to dedicated hardware which can relay the processing results back to the drone for further action. However, the reliance on this feature might hamper the drone operation in the case of lost internet connectivity. An emerging trend in drone communication is the use of cloud computing or the Internet of Things (IoT) to augment drone operation. This concept has been proposed in the Industrial IoT (IIoT) field such as for optimal power line inspection using UAVs [40]. Some architectures have been developed with the focus of getting connected using resourceless or resource-constrained connectivity such as drone-assisted vehicular networks (DAVN) or flying ad-hoc networks (FANET) for IoT-enabled scenarios [41,42], giving rise to the concept of internet of drones (IoD) [43,44]. This trend has been observed elsewhere in applications requiring transmitting environmental sensor data for constant monitoring, as demonstrated in [45,46] and is expected to have significant integration in the field of crowd monitoring as well.

2.4. Power Management

Whereas drones can perform a task beyond human imagination, the current technology that relies on onboard batteries introduces some limitations to their operation. Specifically, the flight time is constrained to only tens of minutes. The additional payload carried by the drone will further reduce its flight time. Such observations have been described in [28], where a DJI Matrice UAV has shown a reduction of nearly 46% of its flight time when carrying an additional 320 g of camera payload [28]. A survey of the literature has indicated two strategies used in enhancing the drone flight time.
Firstly, the use of alternative power supply, allowing additional power for flight without the frequent need for recharging. Secondly, the use of a charging station (CS) either on the ground or on top of a building or vehicle that employs various technologies to recharge an operational drone. The former method is used in the commercial Perimeter 8 multirotor drone (Skyfront), which operates on a hybrid gasoline-electric engine system and has demonstrated around 13 h of flight time over California’s Coastal Range [47]. Similarly, the Hybrix 2.1 (Quarternium) multirotor drone has demonstrated over 10 h of flight time using a petrol-electric fuel-injection engine [48]. Whereas the use of an advanced engine propulsion system for drone provides enhanced endurance, it will also significantly increase the cost of these drones. However, the availability of these technologies for multi-rotor drones offers promising prospects in crowd monitoring and analyses applications.
Other works have employed the charging station (CS) strategies allowing conventional drones to ensure the continuity of operations. One method discussed in [49] automated the swapping of multiple drones to ensure continuity of operation by identifying when a drone needs a recharge, deploying another one to its place, and directing the low-powered drone to a ground CS. The charging is to be conducted contactless to ensure complete automation, removing the need for an operator to remove and install the battery manually. In a similar concept, instead of a fixed ground CS, Trotta et al. [28] has described the use of a mobile CS for continuous crowd tracking, in which scheduling of the drone charging can be made dynamically by changing the location of the CS to follow the drone position, yielding higher efficiency and continuous connectivity of service.
Another method discussed has been the ‘battery hot-swapping’, in which a battery on the drone is switched without needing to turn the drone off at the CS (for example, using a robotic arm). In this case, multiple battery backups are made available instead of multiple drones [50]. However, an operation time-gap exists while the hot swapping takes place. Other proposed methods include contactless drone charging from the CS using laser beam technology [51,52] while the drone is mid-air. In the context of crowd monitoring, this method will require utmost operational safety concern since the use of high-intensity lasers can prove hazardous to human health and can impose disturbance in urban living areas [53]. Photovoltaic (PV) cells harvesting solar energy have also been investigated in the literature to allow charging without the need of requiring the landing maneuvers. However, this technique depends on having adequate solar irradiation, and is mainly suitable for fix-wing typed drones due to the vast surface area available on the drone body for attaching the PV cells [53]. For continuous aerial surveillance, one alternative method suggested in the literature is via a tethered drone system [54,55,56]. In this method, a drone is powered from a ground station using generators or battery packs, effectively providing unlimited endurance [57]. In addition, the tethering can provide an additional secured communication uplink/downlink channel. A significant drawback of this method is the apparent cable length restriction which subsequently imposes restrictions on the drone coverage area. One improvement suggested to overcome this limitation is by introducing a network of drones chained via tethering cables with control mechanisms implemented on the ground station [58]. Figure 1 summarizes the drone power management strategies.
Since adding payloads such as cameras and recording equipment will inevitably reduce the drone’s flight time, efficient power management is necessary for monitoring and analysis applications. Table 1 summarizes the drone architecture as discussed in this section.

3. Applications and Algorithms

This section will cover the applications of drones in the area of crowd monitoring and analysis. In addition, various algorithms used for achieving the said purpose will be presented, along with the emerging challenges.

3.1. Crowd Detection and Monitoring

Drones are increasingly utilized alongside conventional CCTV crowd monitoring and surveillance devices and systems. Crowd monitoring can be an essential tool in public safety, since an accurate crowd assessment via monitoring can avoid potential disasters caused by abnormal crowd formation [18]. The mobility of drones overcomes the limitations of imaging angle, coverage and deployment cost of conventional CCTVs [5,18], thus increasing its significance in real-time surveillance operations.
The authors in [18] have proposed using RGB and thermal imagery data for crowd monitoring with a single drone. The authors have argued that since visuals captured by drones are most likely nadir imagery (picture taken vertically or from aerial perspective), the approach of identifying face and extracting body features is not suitable. Additionally, whereas RGB imagery provides excellent details due to higher resolution, detecting the crowd is a challenging task (earlier research such as presented in [61] used color-coded participants for person detection), which can benefit from additional input such as provided by a lower resolution thermal imagery. A similar approach has been employed in [19,21], in which thermal imagery alone has been used to detect the human crowd. The authors have proposed a detection method based on Region of Interest (ROI) extraction and Supervised Machine Learning (SML) classification. Although promising results have been presented, the techniques have only been verified in a reduced number of sample images. In [24], the authors have presented a Multiview CNN algorithm, which takes an RGB input coupled with artificially generated crowd heat maps. The authors have proposed that the algorithm can be implemented on a processing board available on commercial drones such as the NVIDIA Jetson TX2 board, ensuring its feasibility for real-time aerial crowd detection tasks. In addition to detection, the authors in [37] have presented a proof of concept for real-time crowd localization methods using the onboard GPS sensor of a drone. This enables for plotting the localization information in a mapping software such as Google Earth.

3.2. Crowd Size Estimation

Crowd size estimation is an integral part of crowd management, as the crowd size can indicate potential risk emerging from its participants’ behavior. One method for crowd size estimation is crowd counting. Several studies have described methods for counting crowds, each addressing different potential challenges to perform precise counting from images obtained through aerial imagery. One of these challenges is the scale variations of crowd images obtained from the drones. Since local regulations in some countries do not allow drones flying directly above the crowds, aerial images tend to be taken at an angle; therefore, they are susceptible to scale variations due to changes in the drone’s height. In [8], the authors have presented a method dubbed scale-adaptive real-time crowd detection and counting method for drone images (SARCCODI) for counting dense crowd which considers the images’ scaling factors. The method has been proven to outperform Convolutional Neural Network (CNN)-based methods for a similar aerial image datasets. Whereas many advanced crowd counting algorithms focus on single modal data (RGB), recently, the authors in [23] have presented a different approach by introducing a multi-modal data (RGB and thermal) CNN to improve performance and accuracy. The authors have created a public dataset called DroneRGBT consisting of 3600 RGB and thermal image pairs, and have devised an algorithm that can work separately on either RGB or thermal images, or with the fusion of the two data sets. It is worth noting that within the limited works for drone images in crowd counting applications, very few have offered perspectives into the ability of the proposed method to work in real-time. Specifically, whether the computational power required to achieve the crowd counting is lightweight enough to be embedded onboard a drone and executed during the drone operation. Both works in [7,8] have discussed a lightweight approach to crowd counting has been tailored to processing tasks in real-time.
In addition to crowd counting, crowd density estimation is also an interesting focus for crowd size estimation. Earlier crowd density estimation methods have relied on detection-based approaches, regression-based approaches or density estimation-based approaches [62]. In [17], the authors described using color-based segmentation to perform crowd density estimation. The method is beneficial in aerial images where the color of the crowd is very distinguishable from the background such as in the Hajj pilgrim images. In further work, the authors have applied a modified corner detection algorithm called the Features from Accelerated Segment Test (FAST) for density estimation in aerial crowd images [63]. The method, however, is more susceptible to misdetection due to different lighting conditions. A more recent approach into this field has centered on the use of CNN-based methods. Specifically for drone imagery, the authors in [26] have described employing an algorithm to estimate crowd density accurately from drone images with perspective distortion by feeding a perspective map into a neural network (CSRNet). This has been achieved by moving away from estimating crowd density on the image plane to estimation on the head plane, a plane that is parallel to the ground and translated vertically to a height of an average person. Figure 2 shows an example of crowd detection from images carried out using CNN method.

3.3. Crowd Tracking

Tracking crowd movement can provide insight into potential risks caused by the crowd dynamics, such as the emergence of riots [29] that can prove to be an essential tool in surveillance operations. Optical flow, an algorithm that estimates the motion of image intensities temporally in a video sequence [65], has been used in motion estimation applications in the past. Whereas the high computation requirement of the optical flow method is now achievable using single board computers, real-time tracking application remains a challenge. Authors in [60] argue that the traditional optical flow methods fail to perform since drone imagery that is often susceptible to change in viewing angles and perspective due to camera movements during flight. The authors have proposed increasing the efficiency and robustness of moving target tracking using optical flow by compensating the camera motion, removing the moving background and then segmenting the independently moving foreground blobs or other similar hindering means. In [32], a different approach has been employed to perform tracking of the human crowd from the drone datasets. The authors have applied globally optimal greedy algorithms that have been first introduced in [66] to estimate the tracks of multiple targets in the captured video sequence. In another work, the authors in [67] have proposed a traverse order generation scheme to address periodical surveillance of multiple moving targets. The proposed scheme uses multiple drones in a decentralized manner. The scheme is designed in a way that can determine the sequence of tracked targets. All the targets are visited sequentially by predicting the position of the next target in path planning. As targets move from one area to another, the presented algorithm switch between drones. The proposed strategy provides an excellent opportunity for the crowd monitoring system to detect abnormal activities. It is worth mentioning that datasets tailored for tracking applications are pretty scarce, with the Visdrone [68] dataset remaining the most extensive dataset tailored for multi-object tracking, apart from the smaller UAVDT dataset [69]. It has recently been utilized for the tracking of individuals in crowds not complying with the COVID-19 SOPs [70] as shown in Figure 3.

3.4. Crowd Analyses

An emerging trend in aerial imagery is analyzing crowd to identify anomalies or changes in crowd behaviors and structures. In [61], an analysis of pedestrian crowd behavior has been presented. The authors have described using the complex event detector method proposed in [71] to identify the behavior of pedestrian groups in multiple scenarios such as corridor walk, bottleneck pathway and escape situation. Authors in [30] described an approach to identify abnormal crowd behaviors using drone-based thermal imagery based on two factors; the crowd density and its velocity. A truth table containing a combination of velocity factors and density factors will decide whether the crowd will be classified as normal or abnormal. For example, an abnormal crowd is anticipated if the crowd’s velocity increases and its density decreases. However, if both the crowd velocity and density decrease, then the crowd is considered normal. Corner detection and optical flow-based algorithm have been employed to identify both factors in a crowd and a multi-task CNN is employed to carry out crowd detection and density estimation. The authors have also demonstrated the feasibility of real-time detection with processing carried out using the single-board computers onboard the drone. The status of crowd behavior analysis is then displayed on the drone handheld controller, as shown in Figure 4.
Another field of interest is the live analysis of crowd behavior during surveillance operation. The concept is demonstrated in [31] where the authors have described a drone surveillance system to identify violent individuals in a crowd. This has achieved using human pose estimation carried out via the proposed ScatterNet Hybrid Deep Learning (SHDL) network algorithm. A custom dataset has been curated from aerial imagery of individuals performing one of the five violent activities (punching, stabbing, shooting, kicking, and strangling). In contrast to similar work in [72], the authors in [31] have employed cloud computing to perform resource-intensive pose estimation tasks, allowing the system to be implemented in real-time applications. Authors in [73] have proposed a priority-based routing framework for flying adhoc networks (PRoFFAN) for better data delivery to decision and control centers. They have found that the PRoFFAN has improved the response time of the flying adhoc network (FANET) application. This improvement is achieved by prioritizing the sending and forwarding of critical image data from the UAV to the control center. Regarding of drones’ applications in crowd management, delivering such important image features as early as possible will save lives and enhance the crowd’s safety and flow.

4. Privacy, Safety and Ethical Implications

4.1. Legal Frameworks

As no agreed standard and regulation exists across the globe, each country has specified its rules regarding the use of drones. In a survey comparing drone legislation among thirty-five (35) Organization for Economic Co-operation and Development (OECD) countries, it has been found that 89% (31 countries) impose flight restrictions in overcrowded areas such metropolitan cities [74]. Other restrictions include flight distance restrictions (35 countries) and distance from building restrictions (23 countries) [74]. In general, regulations are made related to several key characteristics: for example, the size and weight of the drones, flight altitude, flight purpose, and restriction requirements [74]. Pilot licensing, ownership registration, and insurance are also common elements included in the contents of drone legislation [75]. Drone legislation also varies according to the types of drones used, whether for recreational or commercial reasons. The approach to legislation related to commercial drones is more stringent with some countries imposing total or effective bans to their use, whereas others set restrictive standards such as line of sight requirements while operating drones [75].
It is worth noting that these legislations may change over time, especially since commercial drones are nowadays widely used in business, law enforcement, and for environmental monitoring purposes [76]. For example, the European Union has issued a regulation enforceable from 31December 2020, that mandates the registration of drones and no longer differentiates between commercial and recreational drone activities, but instead considers a drones’ weight, their specifications and the intended operation [77]. Restrictions on dealing with the crowds also differ by legislation. European Union law prohibits uncertified drones from flying over crowds, where an assembly of people is defined based on peoples’ movement in a space, not the number of people [78]. In the United States, recent rules changes (April 2021) allow a small drone to be flown over people if it weighs less than 0.55 pounds (~250 g), with shielded rotating parts and the drone is equipped with Remote ID, a technology that allows drones to be identified from the ground [79]. Overall, legislation around the use of drones is part of the ongoing efforts in many countries and is likely to undergo drastic changes in the coming years in response to the increased use of drones worldwide in areas such as business, logistics, agriculture, and weather, or animal tracking and monitoring applications.

4.2. Safety Considerations

Although drones are useful during crowd interactions, they can also pose risks and threats with potential health implications, and even fatalities during the operation. In the United States, between 2010 and 2017, injuries related to amateur aircraft including drones experienced a sharp increase, with more than 270 cases requiring hospitalization [80]. Although predominantly reported in adults, the reported incidences of injuries have included children under 18 years of ages [81]. Injuries caused by drones can be due to an impact which can cause contusion or caused by the rapidly-rotating propellers and rotors, harming different body parts due to lacerations [82]. In addition, more severe cases such as skull fracture [83], severe eye injury [84], and organ damage due to exposure to chemicals [81] have been reported. In maintaining the safe navigation of drones around the crowd, experts have recommended using protective equipment and tools, especially for the body parts most susceptible to injuries such as hands and heads. In addition, propeller guards can be installed to avoid injury [80,82] during retrieval and in the event of an unexpected landing. A small number of works have suggested using technological advancements to avoid fatal drone-crowd interactions. The authors in [85] have described a deep learning method of a fully convolutional network for crowd identification from drone images that could output a heat map of safe or non-safe landing zones based on the presence of crowd and pedestrian in a specific area. This work is improved from the previous results by the authors in [20,86], where similar method has been used to identify no-fly zones, such as the areas populated by the crowd and adjusting the drone flight planning according to the local legal requirements. In [87], an innovation in terms of drone hardware is proposed for cargo drone applications. The authors argued that whereas traditional propeller guards provide safety measures, they introduce performance penalties to the drone operation. The authors have proposed a drone design with morphing arms that can be retracted during take-off and landing and extended during flight time. In addition, an 8 mm opening on the cage structure has been selected to prevent children’s fingers from passing through it [87]. It should be noted that whereas some jurisdictions have drafted their regulations, there is no international standard for the airworthiness of drones, which has caused a lot of drone incidents during events involving crowd presence [88]. One specific measure proposed by the authors of [88] is the availability of a redundancy system in drones that provides a fail-safe mechanism capable of dealing with the loss of control or communication and subsequently providing safe landing. A combination of advances in a drone’s hardware and software capabilities can provide safety and minimize risks during the operation near crowds.

4.3. Privacy and Ethical Implications

As the use of drones expands from the military domains to recreational and commercial spaces, the privacy and techno-ethical discussion surrounding drone use in crowd management are justified. The availability of visual capture devices even on low-cost hobbyist drones opens the possibility of abuse [89]. Recent estimates have shown that as much as 30% of people have negative feelings (nervous, angry, sad) upon seeing a drone flying close to their living place [90]. Drones have already been used for violating individual space and privacy, especially against female targets through aerial photography and video recording of their private lives by malicious users [76,91]. In a crowded environment, data collection in the form of visual images and facial recognition poses serious privacy concerns. The authors in [89] have discussed that the best privacy practice (for example, the one proposed in [92]) recommends adequately addressing data collection, data sharing, data storage, data security, and data use policies. However, such practice currently remains as a recommendation rather than being enforced via as legislation. In the case of privacy breaches, challenges arise, among others in determining the drone ownership, establishing intent, and arguing the case for reasonable expectation to privacy [93]. One study investigating the relationship between drone altitude and the success rate of privacy violation attacks from drone cameras indicated that regulations relating to use of drones to a specific height can help reduce the identifiable features from the recorded images [94]. This, of course, depends greatly on the type of cameras available onboard the drone and varies significantly from one drone to another. Other works have suggested that anonymization techniques such as blurring of images can be applied to the collected image data to remove identifiable information from the images, thus minimizing the risk of privacy violation [89,95]. However, as noted in [96], anonymization is often insufficient if the drones also collect information that extends beyond visual data, such as geolocation and landmarks, which could still point to an identifiable details when combined with visual data. Drone detection and deterrence techniques have also been suggested to counter illegal and obtrusive use of drones, especially in sensitive public spheres such as airports, medical facilities and crowded areas. Current technologies proposed include the use of radar-based detection, visual detection, acoustic detection, and radio frequency detection, as extensively reviewed in [97]. In addition to single modality detection, some research have also investigated multiple modalities or hybrid detection techniques such as visual combined with acoustic such as presented in [98] Whereas many of these techniques addresses single drone detection, research in [99,100] proposed radar-based methods for identifying drones in swarm formation. Drone detection techniques can assist authorities and enforcement agencies to interdict and intervene for authoritative prohibition of unlawful drone operations.
Another area of concern regarding the ethical considerations of using a drone over the crowd is the militarization of drones. While amateur and commercial drones are inherently non-violent, drones used in waring zones pose severe ethical issues for civilian crowds, including pedestrians and bystanders. Although drone strikes are often described as “precise”, thus minimizing risk and casualties, but the reality on the ground is not as transparent as it is claimed. For example, drone strikes have been reported to cause more severe amputation and most traumatic among the civilian (called collateral damage in military terms) than other modern weapons [101]. In addition, in contrast to minimal risk as suggested in [102], sources indicated that civilian casualties are far more damaging than what the governments and authorities have reported with little or unreliable transparency and accountability [76,103].

5. Outlook and Conclusions

Drones are becoming the modern state-of-the-art solutions for crowd management and continuous tracking. Current research directions are mainly focused on single-drone deployment for crowd detection, localization, tracking and prediction. However, swarm drone arrangement is expected to find its way in future crowd management, coupled with emerging technologies such as block-chain and 5G connectivity [104,105]. Among the challenges to be addressed are optimal routing and the autonomous distribution of drones to perform crowd monitoring and tracking [106]. For instance, when managing the recent COVID-19 pandemic, such swarm arrangements have been proposed to perform healthcare and policy compliance surveillance over a crowded population [107,108,109].
Due to occlusion imposed by varying weather conditions, scale changes, and viewpoint variations, aerial images are challenging to analyze compared with datasets obtained from traditional fixed-camera imaging methods. For instance, as noted in [110,111], the performance of modern algorithms on applications such as crowd-counting and multiple objects tracking algorithms on challenging aerial datasets remain unsatisfactory and offers a large room for improvements. In terms of real-time applications for crowd analysis, two major key enablers are identified. Firstly, the availability of high-capacity single board computers which exist in small form, are allowing their deployments onboard modern drones alongside visuals and other telemetry sensors. Secondly, high-speed internet connectivity will allow image analysis to be carried out on remote servers or on the cloud, practically eliminating the hardware barriers for implementing algorithms requiring high computational power. Both methods have been demonstrated in the surveyed literature and are expected to become significant in future applications requiring real-time capability.
As the requirement for drone ownership becomes complete due to low-cost and high market availability, it is expected that drone legislation will constantly change to adapt to the challenges surrounding safety, privacy, and ethical issues for their use. From drone ownership registration to piloting license requirements, countries with high drone usage have already introduced legislation that will safeguard the community. Standardization might also become the focus of attention in the future where a drone operator who has obtained a license in one country is recognized in other countries within a specific union, alliance or economic zones. Ethical issues involving data collection of identifiable individuals in crowd settings will require immediate attention preventing thus drones from abuse. Among future directions for ethical and legal discussion is the issue of drone autonomy, specifically in applications involving crowd analysis or crowd prediction. For example, classifying an individual or a group of people as violent or non-violent based on their limb orientation or body postures presents concern regarding misclassification and potential abuse. Another major issue of interest is when a drone is tasked with making a life-altering decision and acts upon it autonomously. While seemingly futuristic, a recent publication by the United Nations Security Council has already reported an instance where a lethal drone equipped with Artificial Intelligence has been programmed to autonomously track and attack targets without communication with its operator [112]. Recent accepted work on the interdiction of SOP is pointing to the futuristic use of the drones in pandemics [112]. Policy discussion around the enormous issues presented in such a situation is currently inadequate due to the clandestine and covert nature of technological use in warfare, albeit the lack of transparency and accountability.
In summary, this paper has provided a comprehensive review of drones from a crowd monitoring and analysis perspective. Related drone architectures including the drone hardware, power management, onboard sensors, and communications have been discussed. Recent work focusing on crowd related applications such as crowd detection, monitoring, volume estimation, tracking, and analysis has indicated an increased interest in this field, particularly due to ease of access to drones for research and increased demands for their use in public security and safety applications. It is worth noting that the legal framework and policies regarding the use of drones are lag behind the rapid adoption of their use world-wide. As the rationale technical capabilities of the drone develop even more featured, our understanding of its impact on crowd management and interaction will help shape future policies, harnessing its benefits while protecting society from its threats and abuse.

Author Contributions

Conceptualization, M.A.H. and S.K.; methodology, M.A.H., W.A., Z.Z.A., Y.M.M., K.K., S.H., M.I. and S.K.; writing—original draft preparation, M.A.H. and S.K.; writing—review and editing, M.A.H., W.A., Z.Z.A., Y.M.M., K.K., S.H., M.I. and S.K.; project administration, W.A. and Z.Z.A.; Supervision, Z.Z.A. and Y.M.M.; funding acquisition, W.A. and S.K. All authors have read and agreed to the published version of the manuscript.

Funding

The authors extend their appreciation to the Deputyship for Research and Innovation, Ministry of Education in Saudi Arabia for funding this research work through the project number QURDO001, titled: “Intelligent Real-Time Crowd Monitoring System Using Unmanned Aerial Vehicle (UAV) Video and Global Positioning Systems (GPS) Data”.

Acknowledgments

The authors extend their appreciation to the technical support they have received from using the laboratories of the Faculty of Engineering, IIUM (Malaysia), Electrical Technology Section, British Malaysian Institute, Universiti Kuala Lumpur (Malaysia), Department of Electrical Engineering, Onaizah College of Engineering and Information Technology, Onaizah Colleges (Saudi Arabia), and Department of Information Technology, Qassim University (Saudi Arabia).

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Alaska, Y.A.; Aldawas, A.D.; Aljerian, N.A.; Memish, Z.A.; Suner, S. The impact of crowd control measures on the occurrence of stampedes during Mass Gatherings: The Hajj experience. Travel Med. Infect. Dis. 2017, 15, 67–70. [Google Scholar] [CrossRef] [PubMed]
  2. Illiyas, F.T.; Mani, S.K.; Pradeepkumar, A.P.; Mohan, K. Human stampedes during religious festivals: A comparative review of mass gathering emergencies in India. Int. J. Disaster Risk Reduct. 2013, 5, 10–18. [Google Scholar] [CrossRef]
  3. Memish, Z.A.; Steffen, R.; White, P.; Dar, O.; Azhar, E.I.; Sharma, A.; Zumla, A. Mass gatherings medicine: Public health issues arising from mass gathering religious and sporting events. Lancet 2019, 393, 2073–2084. [Google Scholar] [CrossRef]
  4. Skogan, W.G. The future of CCTV. Criminol. Public Policy 2019, 18, 161–166. [Google Scholar] [CrossRef] [Green Version]
  5. Tang, Y.; Miao, Y.; Barnawi, A.; Alzahrani, B.; Alotaibi, R.; Hwang, K. A joint global and local path planning optimization for UAV task scheduling towards crowd air monitoring. Comput. Netw. 2021, 193, 107913. [Google Scholar] [CrossRef]
  6. Yuan, Y.; Wang, Z.; Li, M.; Son, Y.-J.; Liu, J. DDDAS-Based Information-Aggregation for Crowd Dynamics Modeling with UAVs and UGVs. Front. Robot. AI 2015, 2. [Google Scholar] [CrossRef] [Green Version]
  7. Castellano, G.; Castiello, C.; Mencar, C.; Vessio, G. Crowd Counting from Unmanned Aerial Vehicles with Fully-Convolutional Neural Networks. In Proceedings of the 2020 International Joint Conference on Neural Networks (IJCNN 2020), Glasgow, UK, 19–24 July 2020; IEEE: Piscataway, NJ, USA, 2020; pp. 1–8. [Google Scholar]
  8. Kuchhold, M.; Simon, M.; Eiselein, V.; Sikora, T. Scale-Adaptive Real-Time Crowd Detection and Counting for Drone Images. In Proceedings of the 2018 25th IEEE International Conference on Image Processing (ICIP 2018), Athens, Greece, 7–10 October 2018; pp. 943–947. [Google Scholar]
  9. Panday, U.S.; Pratihast, A.K.; Aryal, J.; Kayastha, R.B. A review on drone-based data solutions for cereal crops. Drones 2020, 4, 41. [Google Scholar] [CrossRef]
  10. Butcher, P.A.; Colefax, A.P.; Gorkin, R.A.; Kajiura, S.M.; López, N.A.; Mourier, J.; Purcell, C.R.; Skomal, G.B.; Tucker, J.P.; Walsh, A.J.; et al. The drone revolution of shark science: A review. Drones 2021, 5, 8. [Google Scholar] [CrossRef]
  11. Budiharto, W.; Chowanda, A.; Gunawan, A.A.S.; Irwansyah, E.; Suroso, J.S. A Review and Progress of Research on Autonomous Drone in Agriculture, Delivering Items and Geographical Information Systems (GIS). In Proceedings of the 2019 2nd World Symposium on Communication Engineering (WSCE), Nagoya, Japan, 20–23 December 2019; pp. 205–209. [Google Scholar]
  12. Mogili, U.R.; Deepak, B.B.V.L. Review on Application of Drone Systems in Precision Agriculture. In Proceedings of the Procedia Computer Science; Elsevier B.V.: Amsterdam, The Netherlands, 2018; Volume 133, pp. 502–509. [Google Scholar]
  13. Esposito, M.; Crimaldi, M.; Cirillo, V.; Sarghini, F.; Maggio, A. Drone and sensor technology for sustainable weed management: A review. Chem. Biol. Technol. Agric. 2021 81 2021, 8, 1–11. [Google Scholar] [CrossRef]
  14. Wu, D.; Li, R.; Zhang, F.; Liu, J. A review on drone-based harmful algae blooms monitoring. Environ. Monit. Assess. 1914 2019, 191, 1–11. [Google Scholar] [CrossRef] [PubMed]
  15. Shahmoradi, J.; Talebi, E.; Roghanchi, P.; Hassanalian, M. A comprehensive review of applications of drone technology in the mining industry. Drones 2020, 4, 34. [Google Scholar] [CrossRef]
  16. Saraf, V.; Senapati, L.; Swarnkar, T. Application and Progress of Drone Technology in the COVID-19 Pandemic. 2020. Available online: https://www.taylorfrancis.com/chapters/edit/10.1201/9781003137481-4/application-progress-drone-technology-covid-19-pandemic-vasundhara-saraf-lipsita-senapati-tripti-swarnkar (accessed on 18 September 2021).
  17. Almagbile, A. Detecting And Estimating The Levels of Crowd Density From UAV Imagery. Dirasat Hum. Soc. Sci. 2019, 46, 294. [Google Scholar] [CrossRef]
  18. Schulte, S.; Hillen, F.; Prinz, T. Analysis of combined UAV-based RGB and thermal remote sensing data: A new approach to crowd monitoring. In International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences-ISPRS Archives; Copernicus GmbH: Göttingen, Germany, 2017; Volume 42, pp. 347–354. [Google Scholar]
  19. Xiao, Y.; Zheng, H.; Yu, W. Automatic Crowd Detection Based on Unmanned Aerial Vehicle Thermal Imagery. In Advances in Intelligent Systems and Computing; Springer: Berlin/Heidelberg, Germany, 2018; Volume 690, pp. 510–516. [Google Scholar]
  20. Tzelepi, M.; Tefas, A. Graph Embedded Convolutional Neural Networks in Human Crowd Detection for Drone Flight Safety. IEEE Trans. Emerg. Top. Comput. Intell. 2019, 5, 191–204. [Google Scholar] [CrossRef]
  21. Xiao, Y.H.; Zhen, H. Pedestrian Crowd Detection Based Unmanned Aerial Vehicle Infrared Imagery. Appl. Mech. Mater. 2017, 873, 347–352. [Google Scholar] [CrossRef]
  22. Wang, Z.; Li, M.; Khaleghi, A.M.; Xu, D.; Lobos, A.; Vo, C.; Lien, J.M.; Liu, J.; Son, Y.J. DDDAMS-based crowd control via UAVs and UGVs. Procedia Comput. Sci. 2013, 18, 2028–2035. [Google Scholar] [CrossRef] [Green Version]
  23. Peng, T.; Li, Q.; Zhu, P. RGB-T Crowd Counting from Drone: A Benchmark and MMCCN Network. In Proceedings of the Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Springer: Berlin/Heidelberg, Germany, 2021; Volume 12627, pp. 497–513. [Google Scholar]
  24. Castellano, G.; Castiello, C.; Cianciotta, M.; Mencar, C. Multi-view Convolutional Network for Crowd Counting in Drone-Captured Images. Springer 2020, 12538 LNCS, 588–603. [Google Scholar] [CrossRef]
  25. Balbin, J.R.; Garcia, R.G.; Fernandez, K.E.D.; Golosinda, N.P.G.; Magpayo, K.D.G.; Velasco, R.J.B. Crowd counting system by facial recognition using Histogram of Oriented Gradients, Completed Local Binary Pattern, Gray-Level Co-Occurrence Matrix and Unmanned Aerial Vehicle. In Proceedings of the Third International Workshop on Pattern Recognition, Jinan, China, 26–28 May 2018; p. 58. [Google Scholar]
  26. Liu, W.; Lis, K.; Salzmann, M.; Fua, P. Geometric and Physical Constraints for Drone-Based Head Plane Crowd Density Estimation. In Proceedings of the 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Macau, China, 3–8 November 2019; pp. 244–249. [Google Scholar]
  27. Choi-Fitzpatrick, A.; Juskauskas, T. Up in the Air: Applying the Jacobs Crowd Formula to Drone Imagery. Proc. Procedia Eng. 2015, 107, 273–281. [Google Scholar] [CrossRef] [Green Version]
  28. Trotta, A.; Muncuk, U.; Di Felice, M.; Chowdhury, K.R. Persistent Crowd Tracking Using Unmanned AerIal Vehicle Swarms: A Novel Framework for Energy and Mobility Management. IEEE Veh. Technol. Mag. 2020, 15, 96–103. [Google Scholar] [CrossRef]
  29. Müller, T.; Müller, M. Vision-based drone flight control and crowd or riot analysis with efficient color histogram based tracking. In Proceedings of the Airborne Intelligence, Surveillance, Reconnaissance (ISR) Systems and Applications VIII, Orlando, FL, USA, 25 May 2011; Volume 8020R, p. 80200. [Google Scholar]
  30. Shao, Y.; Li, W.; Chu, H.; Chang, Z.; Zhang, X.; Zhan, H. A multitask cascading cnn with multiscale infrared optical flow feature fusion-based abnormal crowd behavior monitoring UAV. Sensors 2020, 20, 5550. [Google Scholar] [CrossRef]
  31. Singh, A.; Patil, D.; Omkar, S.N. Eye in the sky: Real-time drone surveillance system (DSS) for violent individuals identification using scatternet hybrid deep learning network. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. Work. 2018, 2018, 1629–1637. [Google Scholar] [CrossRef] [Green Version]
  32. Wen, L.; Du, D.; Zhu, P.; Hu, Q.; Wang, Q.; Bo, L.; Lyu, S. Drone-based Joint Density Map Estimation, Localization and Tracking with Space-Time Multi-Scale Attention Network. arXiv 2019, arXiv:1912.01811. [Google Scholar]
  33. Hassanalian, M.; Abdelkefi, A. Classifications, applications, and design challenges of drones: A review. Prog. Aerosp. Sci. 2017, 91, 99–131. [Google Scholar] [CrossRef]
  34. Drone Types: Multi-Rotor, vs. Fixed-Wing vs. Single Rotor vs Hybrid VTOL|AUAV. Available online: https://www.auav.com.au/articles/drone-types/ (accessed on 4 August 2021).
  35. Zaludin, Z.; Harituddin, A.S.M. Challenges and Trends of Changing from Hover to Forward Flight for a Converted Hybrid Fixed Wing VTOL UAS from Automatic Flight Control System Perspective. In Proceedings of the 2019 IEEE 9th International Conference on System Engineering and Technology (ICSET), Shah Alam, Malaysia, 7 October 2019; pp. 247–252. [Google Scholar]
  36. The U.S. China Trade War and DJI’s Drone Market Share–DRONELIFE. Available online: https://dronelife.com/2021/03/05/has-the-u-s-china-trade-war-changed-djis-drone-market-share-the-latest-from-drone-industry-insights/ (accessed on 4 August 2021).
  37. Bhattarai, N.; Nakamura, T.; Mozumder, C. Real Time Human Detection and Localization Using Consumer Grade Camera and Commercial UAV. Preprints 2018. [Google Scholar] [CrossRef]
  38. Koubaa, A.; Allouch, A.; Alajlan, M.; Javed, Y.; Belghith, A.; Khalgui, M. Micro Air Vehicle Link (MAVlink) in a Nutshell: A Survey. IEEE Access 2019, 7, 87658–87680. [Google Scholar] [CrossRef]
  39. Dietrich, T.; Andryeyev, O.; Zimmermann, A.; Mitschele-Thiel, A. Towards a Unified Decentralized Swarm Management and Maintenance Coordination Based on MAVLink. In Proceedings of the 2016 International Conference on Autonomous Robot Systems and Competitions (ICARSC), Braganca, Portugal, 4–6 May 2016; pp. 124–129. [Google Scholar]
  40. Zhou, Z.; Zhang, C.; Xu, C.; Xiong, F.; Zhang, Y.; Umer, T. Energy-Efficient Industrial Internet of UAVs for Power Line Inspection in Smart Grid. IEEE Trans. Ind. Inform. 2018, 14, 2705–2714. [Google Scholar] [CrossRef] [Green Version]
  41. Shi, W.; Zhou, H.; Li, J.; Xu, W.; Zhang, N.; Shen, X. Drone Assisted Vehicular Networks: Architecture, Challenges and Opportunities. IEEE Netw. 2018, 32, 130–137. [Google Scholar] [CrossRef]
  42. Lin, C.; He, D.; Kumar, N.; Choo, K.K.R.; Vinel, A.; Huang, X. Security and Privacy for the Internet of Drones: Challenges and Solutions. IEEE Commun. Mag. 2018, 56, 64–69. [Google Scholar] [CrossRef]
  43. Koubaa, A.; Qureshi, B.; Sriti, M.-F.; Javed, Y.; Tovar, E. A service-oriented Cloud-based management system for the Internet-of-Drones. In Proceedings of the 2017 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC), Coimbra, Portugal, 26–28 April 2017; pp. 329–335. [Google Scholar]
  44. Yahuza, M.; Idris, M.Y.I.; Ahmedy, I.B.; Wahab, A.W.A.; Nandy, T.; Noor, N.M.; Bala, A. Internet of Drones Security and Privacy Issues: Taxonomy and Open Challenges. IEEE Access 2021, 9, 57243–57270. [Google Scholar] [CrossRef]
  45. Saha, A.K.; Saha, J.; Ray, R.; Sircar, S.; Dutta, S.; Chattopadhyay, S.P.; Saha, H.N. IOT-based drone for improvement of crop quality in agricultural field. In Proceedings of the 2018 IEEE 8th Annual Computing and Communication Workshop and Conference (CCWC), Las Vegas, NV, USA, 8–10 January 2018; Volume 2018, pp. 612–615. [Google Scholar]
  46. Chae, H.; Park, J.H.; Song, H.N.; Kim, Y.H.; Jeong, H.W. The IoT based automate landing system of a drone for the round-the-clock surveillance solution. In Proceedings of the IEEE International Conference on Advanced Intelligent Mechatronics (AIM), Busan, Korea, 7–11 July 2015; pp. 1575–1580. [Google Scholar]
  47. Diesel-Powered Drone Set New Multi-Day World Record For Longest Flight|Digital Trends. Available online: https://www.digitaltrends.com/cool-tech/drone-unmanned-flight-record/ (accessed on 21 August 2021).
  48. HYBRIX 2.1 Sets New World Record for UAV Flight Time: 10 h, 14 min-UASweekly.com. Available online: https://uasweekly.com/2020/10/26/hybrix-2-1-sets-new-world-record-for-uav-flight-time-10-hours-14-minutes/ (accessed on 21 August 2021).
  49. Williams, A.; Yakimenko, O. Persistent mobile aerial surveillance platform using intelligent battery health management and drone swapping. In Proceedings of the 2018 4th International Conference on Control, Automation and Robotics (ICCAR), Auckland, New Zealand, 20–23 April 2018; pp. 237–246. [Google Scholar]
  50. Galkin, B.; Kibilda, J.; DaSilva, L.A. UAVs as Mobile Infrastructure: Addressing Battery Lifetime. IEEE Commun. Mag. 2019, 57, 132–137. [Google Scholar] [CrossRef] [Green Version]
  51. Nugent, T.; Kare, J. Laser Power for UAVs. 2010. Available online: https://silo.tips/download/lasermotive-white-paper-power-beaming-for-uavs-a-white-paper-by-tj-nugent-and-jt (accessed on 11 September 2021).
  52. Ouyang, J.; Che, Y.; Xu, J.; Wu, K. Throughput Maximization for Laser-Powered UAV Wireless Communication Systems. In Proceedings of the 2018 IEEE International Conference on Communications Workshops (ICC Workshops), Kansas City, MO, USA, 20–24 May 2018; pp. 1–6. [Google Scholar]
  53. Lu, M.; Bagheri, M.; James, A.P.; Phung, T. Wireless Charging Techniques for UAVs: A Review, Reconceptualization, and Extension. IEEE Access 2018, 6, 29865–29884. [Google Scholar] [CrossRef]
  54. Prior, S.D. Tethered Drones for Persistent Aerial Surveillance Applications. 2015. Available online: http://blog.soton.ac.uk/robotics/files/2015/08/Your-article-pages-78-79.pdf (accessed on 11 September 2021).
  55. Liang, X.; Zhao, S.; Chen, G.; Tong, G.; Jiang, L.; Zhang, W. Design and Development of Ground Control System for Tethered UAV. In Proceedings of the 2019 IEEE International Conference on Unmanned Systems (ICUS), Beijing, China, 17–19 October 2019; pp. 291–296. [Google Scholar]
  56. Walendziuk, W.; Falkowski, P.; Kulikowski, K. The Analysis of Power Supply Topologies for Tethered Drone Applications. Proceedings 2020, 51, 25. [Google Scholar] [CrossRef]
  57. Boukoberine, M.N.; Zhou, Z.; Benbouzid, M. Power Supply Architectures for Drones-A Review. In Proceedings of the IECON 2019-45th Annual Conference of the IEEE Industrial Electronics Society, Lisbon, Portugal, 14–17 October 2019; Volume 2019, pp. 5826–5831. [Google Scholar]
  58. Fagiano, L. Systems of Tethered Multicopters: Modeling and Control Design. IFAC-PapersOnLine 2017, 50, 4610–4615. [Google Scholar] [CrossRef]
  59. Al-Sheary, A. Ali Almagbile Crowd Monitoring System Using Unmanned Aerial Vehicle (UAV). J. Civ. Eng. Archit. 2017, 11, 1014–1024. [Google Scholar] [CrossRef] [Green Version]
  60. Minaeian, S.; Liu, J.; Son, Y.J. Effective and Efficient Detection of Moving Targets from a UAV’s Camera. IEEE Trans. Intell. Transp. Syst. 2018, 19, 497–506. [Google Scholar] [CrossRef]
  61. Burkert, F.; Fraundorfer, F. UAV-Based Monitoring of Pedestrian Groups. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2013, XL-1/W2, 67–72. [Google Scholar] [CrossRef]
  62. Sindagi, V.A.; Patel, V.M. A survey of recent advances in CNN-based single image crowd counting and density estimation. Pattern Recognit. Lett. 2018, 107, 3–16. [Google Scholar] [CrossRef] [Green Version]
  63. Almagbile, A. Estimation of crowd density from UAVs images based on corner detection procedures and clustering analysis. Geo-Spatial Inf. Sci. 2019, 22, 23–34. [Google Scholar] [CrossRef]
  64. Khan, A.; Ali Shah, J.; Kadir, K.; Albattah, W.; Khan, F. Crowd Monitoring and Localization Using Deep Convolutional Neural Network: A Review. Appl. Sci. 2020, 10, 4781. [Google Scholar] [CrossRef]
  65. Bovik, A.C. The Essential Guide to Image Processing. 2009. Available online: https://books.google.com.hk/books?hl=zh-CN&lr=&id=6TOUgytafmQC&oi=fnd&pg=PP1&dq=The+Essential+Guide+to+Image+Processing&ots=mahTo8MWsv&sig=kvA7kePjBfH8o_t5_A8_OE4CBI&redir_esc=y#v=onepage&q=The%20Essential%20Guide%20to%20Image%20Processing&f=false (accessed on 7 September 2021).
  66. Pirsiavash, H.; Ramanan, D.; Fowlkes, C.C. Globally-optimal greedy algorithms for tracking a variable number of objects. In Proceedings of the CVPR 2011, Colorado Springs, CO, USA, 20–25 June 2011; pp. 1201–1208. [Google Scholar]
  67. Nagrare, S.R.; Chopra, O.; Jana, S.; Ghose, D. Decentralized Path Planning Approach for Crowd Surveillance using Drones. In Proceedings of the 2021 International Conference on Unmanned Aircraft Systems (ICUAS), Athens, Greece, 15–18 June 2021; pp. 1020–1028. [Google Scholar]
  68. Zhu, P.; Wen, L.; Bian, X.; Ling, H.; Hu, Q. Vision Meets Drones: A Challenge. arXiv 2018, arXiv:1804.07437. Available online: https://arxiv.org/abs/1804.07437 (accessed on 14 September 2021).
  69. Yu, H.; Li, G.; Zhang, W.; Huang, Q.; Du, D.; Tian, Q.; Sebe, N. The Unmanned Aerial Vehicle Benchmark: Object Detection, Tracking and Baseline. Int. J. Comput. Vis. 2020, 128, 1141–1159. [Google Scholar] [CrossRef]
  70. Khel, M.H.K.; Kadir, K.; Albattah, W.; Khan, S.; Noor, M.; Nasir, H.; Habib, S.; Islam, M.; Khan, A. Real-Time Monitoring of COVID-19 SOP in Public Gathering Using Deep Learning Technique. Emerg. Sci. J. 2021, 5, 182–196. [Google Scholar] [CrossRef]
  71. Burkert, F.; Butenuth, M. Complex Event Detection in Pedestrian Groups from UAVs. ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci. 2012, 1, 335–340. [Google Scholar] [CrossRef] [Green Version]
  72. Penmetsa, S.; Minhuj, F.; Singh, A.; Omkar, S.N. Autonomous UAV for suspicious action detection using pictorial human pose estimation and classification. Electron. Lett. Comput. Vis. Image Anal. 2014, 13, 18–32. [Google Scholar] [CrossRef] [Green Version]
  73. Felemban, E.; Sheikh, A.A.; Naseer, A. Improving response time for crowd management in hajj. Computers 2021, 10, 46. [Google Scholar] [CrossRef]
  74. Tsiamis, N.; Efthymiou, L.; Tsagarakis, K.P. A Comparative Analysis of the Legislation Evolution for Drone Use in OECD Countries. Drones 2019, 3, 75. [Google Scholar] [CrossRef] [Green Version]
  75. Jones, T. International Commercial Drone Regulation and Drone Delivery Services. 2017. Available online: https://www.rand.org/content/dam/rand/pubs/research_reports/RR1700/RR1718z3/RAND_RR1718z3.pdf (accessed on 11 September 2021).
  76. Luppicini, R.; So, A. A technoethical review of commercial drone use in the context of governance, ethics, and privacy. Technol. Soc. 2016, 46, 109–119. [Google Scholar] [CrossRef]
  77. Civil drones (Unmanned Aircraft)|EASA. Available online: https://www.easa.europa.eu/domains/civil-drones-rpas (accessed on 10 September 2021).
  78. What is an ‘Assembly of People’?|EASA. Available online: https://www.easa.europa.eu/faq/116553 (accessed on 10 September 2021).
  79. Operations Over People General Overview. Available online: https://www.faa.gov/uas/commercial_operators/operations_over_people/ (accessed on 10 September 2021).
  80. Johnson, J.A.; Svach, M.R.; Brown, L.H. Drone and Other Hobbyist Aircraft Injuries Seen in U.S. Emergency Departments, 2010–2017. Am. J. Prev. Med. 2019, 57, 826–829. [Google Scholar] [CrossRef]
  81. Khan, A.; Brown, L. Recreational Drone-Related Injuries in Children: A Review of National Electronic Injury Surveillance System (NEISS) Data. Cureus 2021, 13, 13–17. [Google Scholar] [CrossRef] [PubMed]
  82. Forrester, M.B. Drone-related injuries treated at emergency departments. Am. J. Emerg. Med. 2019, 37, 2116–2117. [Google Scholar] [CrossRef] [PubMed]
  83. Chung, L.K.; Cheung, Y.; Lagman, C.; Au Yong, N.; McBride, D.Q.; Yang, I. Skull fracture with effacement of the superior sagittal sinus following drone impact: A case report. Child’s Nerv. Syst. 2017, 33, 1609–1611. [Google Scholar] [CrossRef]
  84. Moskowitz, E.E.; Siegel-Richman, Y.M.; Hertner, G.; Schroeppel, T. Aerial drone misadventure: A novel case of trauma resulting in ocular globe rupture. Am. J. Ophthalmol. Case Rep. 2018, 10, 35–37. [Google Scholar] [CrossRef] [PubMed]
  85. Castellano, G.; Castiello, C.; Mencar, C.; Vessio, G. Crowd Detection for Drone Safe Landing Through Fully-Convolutional Neural Networks. In Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); LNCS; Springer: Berlin/Heidelberg, Germany, 2020; Volume 12011, pp. 301–312. ISBN 9783030389185. [Google Scholar]
  86. Tzelepi, M.; Tefas, A. Human crowd detection for drone flight safety using convolutional neural networks. In Proceedings of the 2017 25th European Signal Processing Conference (EUSIPCO), Kos, Greece, 28 August–2 September 2017; pp. 743–747. [Google Scholar]
  87. Kornatowski, P.M.; Feroskhan, M.; Stewart, W.J.; Floreano, D. A Morphing Cargo Drone for Safe Flight in Proximity of Humans. IEEE Robot. Autom. Lett. 2020, 5, 4233–4240. [Google Scholar] [CrossRef]
  88. Waibel, M.; Augugliaro, F. Drone Shows: Creative Potential and Best Practices. 2017. Available online: https://www.research-collection.ethz.ch/bitstream/handle/20.500.11850/125498/eth-50527-01.pdf (accessed on 12 September 2021).
  89. Winkler, S.; Zeadally, S.; Evans, K. Privacy and civilian drone use: The need for further regulation. IEEE Secur. Priv. 2018, 16, 72–80. [Google Scholar] [CrossRef]
  90. Ljungholm, D.P. Regulating government and private use of unmanned aerial vehicles: Drone policymaking, law enforcement deployment, and privacy concerns. Anal. Metaphys. 2019, 18, 16–22. [Google Scholar] [CrossRef] [Green Version]
  91. Thomasen, K. Beyond Airspace Safety: A Feminist Perspective on Drone Privacy Regulation. SSRN Electron. J. 2018, 16. [Google Scholar] [CrossRef]
  92. Telecommunications, N.; Administration, I. Voluntary Best Practices for UAS Privacy, Transparency, and Accountability. 2016. Available online: https://knowbeforeyoufly.org/knowbeforeyoufly/how-to-fly/uas-best-practices (accessed on 20 September 2021).
  93. Scharf, R.L. Drone Invasion: Unmanned Aerial Vehicles and the Right to Privacy. 2019. Available online: https://scholars.law.unlv.edu/facpub/1217/ (accessed on 18 September 2021).
  94. Li, Z.; Gao, C.; Yue, Q.; Fu, X. Toward Drone Privacy via Regulating Altitude and Payload. In Proceedings of the 2019 International Conference on Computing, Networking and Communications (ICNC), Honolulu, HI, USA, 18–21 February 2019; pp. 562–566. [Google Scholar]
  95. Finn, R.L.; Wright, D. Privacy, data protection and ethics for civil drone practice: A survey of industry, regulators and civil society organisations. Comput. Law Secur. Rev. 2016, 32, 577–586. [Google Scholar] [CrossRef]
  96. Finn, R.; Donovan, A. Big Data, Drone Data: Privacy and Ethical Impacts of the Intersection Between Big Data and Civil Drone Deployments. 2016. Available online: https://www.researchgate.net/publication/309182307_Big_Data_Drone_Data_Privacy_and_Ethical_Impacts_of_the_Intersection_Between_Big_Data_and_Civil_Drone_Deployments (accessed on 17 September 2021).
  97. Taha, B.; Shoufan, A. Machine Learning-Based Drone Detection and Classification: State-of-the-Art in Research. IEEE Access 2019, 7, 138669–138682. [Google Scholar] [CrossRef]
  98. Liu, H.; Wei, Z.; Chen, Y.; Pan, J.; Lin, L.; Ren, Y. Drone Detection Based on an Audio-Assisted Camera Array. In Proceedings of the 2017 IEEE Third International Conference on Multimedia Big Data (BigMM), Laguna Hills, CA, USA, 19–21 April 2017; pp. 402–406. [Google Scholar]
  99. Zheng, J.; Chen, R.; Yang, T.; Liu, X.; Liu, H.; Su, T.; Wan, L. An Efficient Strategy for Accurate Detection and Localization of UAV Swarms. IEEE Internet Things J. 2021, 8, 15372–15381. [Google Scholar] [CrossRef]
  100. Zheng, J.; Yang, T.; Liu, H.; Su, T.; Wan, L. Accurate Detection and Localization of Unmanned Aerial Vehicle Swarms-Enabled Mobile Edge Computing System. IEEE Trans. Ind. Inform. 2021, 17, 5059–5067. [Google Scholar] [CrossRef]
  101. Heszlein-Lossius, H.; Al-Borno, Y.; Shaqqoura, S.; Skaik, N.; Melvaer Giil, L.; Gilbert, M.F. Traumatic amputations caused by drone attacks in the local population in Gaza: A retrospective cross-sectional study. Lancet Planet. Health 2019, 3, e40–e47. [Google Scholar] [CrossRef] [Green Version]
  102. Byman, D. Why drones work: The case for Washington’s weapon of choice. Foreign Aff. 2013, 92, 32. [Google Scholar]
  103. Boyle, M.J. The legal and ethical implications of drone warfare. Int. J. Hum. Rights 2015, 19, 105–126. [Google Scholar] [CrossRef]
  104. Xiao, W.; Li, M.; Alzahrani, B.; Alotaibi, R.; Barnawi, A.; Ai, Q. A Blockchain-Based Secure Crowd Monitoring System Using UAV Swarm. IEEE Netw. 2021, 35, 108–115. [Google Scholar] [CrossRef]
  105. Chamola, V.; Hassija, V.; Gupta, V.; Guizani, M. A Comprehensive Review of the COVID-19 Pandemic and the Role of IoT, Drones, AI, Blockchain, and 5G in Managing its Impact. IEEE Access 2020, 8, 90225–90265. [Google Scholar] [CrossRef]
  106. Chen, X.; Tang, J.; Lao, S. Review of Unmanned Aerial Vehicle Swarm Communication Architectures and Routing Protocols. Appl. Sci. 2020, 10, 3661. [Google Scholar] [CrossRef]
  107. Conte, C.; de Alteriis, G.; De Pandi, F.; Caputo, E.; Lo Moriello, R.S.; Rufino, G.; Accardo, D. Performance Analysis for Human Crowd Monitoring to Control COVID-19 disease by Drone Surveillance. In Proceedings of the 2021 IEEE 8th International Workshop on Metrology for AeroSpace (MetroAeroSpace), Naples, Italy, 23–25 June 2021; pp. 31–36. [Google Scholar]
  108. Alsamhi, S.H.; Lee, B.; Guizani, M.; Kumar, N.; Qiao, Y.; Liu, X. Blockchain for Decentralized Multi-Drone to Combat COVID-19. arXiv 2021, arXiv:2102.00969. [Google Scholar]
  109. Kumar, A.; Sharma, K.; Singh, H.; Naugriya, S.G.; Gill, S.S.; Buyya, R. A drone-based networked system and methods for combating coronavirus disease (COVID-19) pandemic. Futur. Gener. Comput. Syst. 2021, 115, 1–19. [Google Scholar] [CrossRef]
  110. Du, D.; Wen, L.; Zhu, P.; Fan, H.; Hu, Q.; Ling, H.; Shah, M.; Pan, J.; Al-Ali, A.; Mohamed, A.; et al. VisDrone-CC2020: The Vision Meets Drone Crowd Counting Challenge Results. In Proceedings of the Computer Vision — ECCV 2020 Workshops; Glasgow, UK, 23–28 August 2020, Bartoli, A., Fusiello, A., Eds.; Lecture Notes in Computer Science; Springer: Cham, Switzerland, 2020; Volume 12538, pp. 675–691. [Google Scholar] [CrossRef]
  111. Fan, H.; Du, D.; Wen, L.; Zhu, P.; Hu, Q.; Ling, H.; Shah, M.; Pan, J.; Schumann, A.; Dong, B.; et al. VisDrone-MOT2020: The Vision Meets Drone Multiple Object Tracking Challenge Results. In Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); LNCS; Springer: Berlin/Heidelberg, Germany, 2020; Volume 12538, pp. 713–727. ISBN 9783030668228. [Google Scholar]
  112. Military Drones May Have Attacked Humans for First Time without Being Instructed to, UN Report Says|The Independent. Available online: https://www.independent.co.uk/life-style/gadgets-and-tech/drone-fully-automated-military-kill-b1856815.html (accessed on 14 September 2021).
Electronics 10 02974 g001 550
Figure 1. Drone power management. (a) Solar-powered drones, suitable for fixed-wing UAVs. (b) Hybrid-powered drones, combining energy from electric and fuel cells. (c) Drone-swapping method, with a fixed charging station. (d) Laser-powered on-the-air recharging. (e) Battery hot-swapping, without powering down the drone. (f) Mobile recharging stations, with programmed charging scheduling, and (g) Tethered drones.
Figure 1. Drone power management. (a) Solar-powered drones, suitable for fixed-wing UAVs. (b) Hybrid-powered drones, combining energy from electric and fuel cells. (c) Drone-swapping method, with a fixed charging station. (d) Laser-powered on-the-air recharging. (e) Battery hot-swapping, without powering down the drone. (f) Mobile recharging stations, with programmed charging scheduling, and (g) Tethered drones.
Electronics 10 02974 g001
Electronics 10 02974 g002 550
Figure 2. An example of crowd size estimation using Deep CNNs [64].
Figure 2. An example of crowd size estimation using Deep CNNs [64].
Electronics 10 02974 g002
Electronics 10 02974 g003 550
Figure 3. Crowd Tracking for COVID-19 SOP breachers [70].
Figure 3. Crowd Tracking for COVID-19 SOP breachers [70].
Electronics 10 02974 g003
Electronics 10 02974 g004 550
Figure 4. Abnormal behavior detection via drone as demonstrated in [30] on a simulated crossroad scene. (a) Normal crowd behavior and (b) Abnormal behavior showing increased crowd velocity and crowd escaping behavior.
Figure 4. Abnormal behavior detection via drone as demonstrated in [30] on a simulated crossroad scene. (a) Normal crowd behavior and (b) Abnormal behavior showing increased crowd velocity and crowd escaping behavior.
Electronics 10 02974 g004
Table
Table 1. Drone architecture in the literature utilized for crowd monitoring and analysis purpose.
Table 1. Drone architecture in the literature utilized for crowd monitoring and analysis purpose.
Make/ModelVisual SensorImage Resolution
(px)		Onboard SensorsCommunicationReference
DJI Phantom 4, DJI Phantom 4 Pro and DJI MavicRGB + Thermal512 × 640 (Thermal)--[23]
Microdrones MD4-1000RGB + Thermal1920 × 1080 (RGB)
320 × 240 (Thermal)		--[18]
DJI Matrice 100RGB---[28]
DJI Phantom 4 ProRGB---[26]
Custom QuadcopterThermal336 × 256 QGroundControl + MAVLink[30]
Custom QuadcopterThermal640 × 512--[21]
Sensefly XbeeRGB--Pix4d UAV[59]
DJI Matrice 100RGB-GPS-[37]
DJI Phantom 4, DJIPhantom 4 Pro, DJI MavicRGB1920 × 1080--[32]
3DR X8+RGB720 × 960 -[60]
Parrot AR 2.02 × RGB1280 × 720 (front facing)
320 × 240 (downward facing)		IMU
Barometric sensor		Cloud computing[31]
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Husman, M.A.; Albattah, W.; Abidin, Z.Z.; Mustafah, Y.M.; Kadir, K.; Habib, S.; Islam, M.; Khan, S. Unmanned Aerial Vehicles for Crowd Monitoring and Analysis. Electronics 2021, 10, 2974. https://doi.org/10.3390/electronics10232974

AMA Style

Husman MA, Albattah W, Abidin ZZ, Mustafah YM, Kadir K, Habib S, Islam M, Khan S. Unmanned Aerial Vehicles for Crowd Monitoring and Analysis. Electronics. 2021; 10(23):2974. https://doi.org/10.3390/electronics10232974

Chicago/Turabian Style

Husman, Muhammad Afif, Waleed Albattah, Zulkifli Zainal Abidin, Yasir Mohd. Mustafah, Kushsairy Kadir, Shabana Habib, Muhammad Islam, and Sheroz Khan. 2021. "Unmanned Aerial Vehicles for Crowd Monitoring and Analysis" Electronics 10, no. 23: 2974. https://doi.org/10.3390/electronics10232974

APA Style

Husman, M. A., Albattah, W., Abidin, Z. Z., Mustafah, Y. M., Kadir, K., Habib, S., Islam, M., & Khan, S. (2021). Unmanned Aerial Vehicles for Crowd Monitoring and Analysis. Electronics, 10(23), 2974. https://doi.org/10.3390/electronics10232974

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop