Rapid Damage Estimation of Texas Winter Storm Uri from Social Media Using Deep Neural Networks

Abstract

The winter storm Uri that occurred in February 2021 affected many regions in Canada, the United States, and Mexico. The State of Texas was severely impacted due to the failure in the electricity supply infrastructure compounded by its limited connectivity to other grid systems in the United States. The georeferenced estimation of the storm’s impact is crucial for response and recovery. However, such information was not available until several months afterward, mainly due to the time-consuming and costly assessment processes. The latency to provide timely information particularly impacted people in the economically disadvantaged communities, who lack resources to ameliorate the impact of the storm. This work explores the potential for disaster impact estimation based on the analysis of instant social media content, which can provide actionable information to assist first responders, volunteers, governments, and the general public. In our prototype, a deep neural network (DNN) uses geolocated social media content (texts, images, and videos) to provide monetary assessments of the damage at zip code level caused by Uri, achieving up to 70% accuracy. In addition, the performance analysis across geographical regions shows that the fully trained model is able to estimate the damage for economically disadvantaged regions, such as West Texas. Our methods have the potential to promote social equity by guiding the deployment or recovery resources to the regions where it is needed based on damage assessment.

1. Introduction

The severe Winter Storm Uri occurred during 11–21 February 2021, causing power outages across the State of Texas. It was declared a national disaster in the US, with the Federal Emergency Management Agency (FEMA) coordinating the federal government response. Texas was severely impacted due to the failure in the electricity supply infrastructure compounded by its limited connectivity to other grid systems in the United States. Loss of electricity supply affected 10 million people directly, with cascading failures in heat, water, transportation, and food supply, among others [1,2]. In the aftermath, the monetary damage assessment across communities is crucial for efficient and equitable allocation of resources for response and recovery. However, this assessment has still not been widely available months after the event, due to the resource-demanding and time-consuming preliminary damage assessment (PDA) procedures including self-reporting, onsite surveys, and insurance claims [3]. Although some insurance organizations, such as the Insurance Council of Texas (ICT), collected data for building and vehicle claims, they did not publish the raw data.

Social media platforms such as Facebook and Twitter allow users to share real-time data including text, image, and video with location information. During disasters, people can facilitate information diffusion, gain situational awareness, and request assistance to enhance disaster response through social media platforms [4,5,6,7,8,9,10]. Such digital platforms can provide a communication channel to disadvantaged communities through the accounts of local residents or volunteers [11,12,13]. For instance, one recent work utilized power outages, pipe bursts, and food accessibility data on Mapbox, SafeGraph, and 311 in Harris Country, Texas, during Uri, and the following analysis revealed that low-income and racial/ethnic minority groups were more disrupted [14]. While social media can be useful in disaster information exchange, the representativeness of such data varies by demographic factors such as age, gender, race, and educational attainment, potentially creating bias in data analysis [15,16]. For instance, younger users in urban areas are more likely to geotag their social media messages [17,18].

The spatial difference in the social media contents from various communities can be used to identify the needs of diverse social groups and measure the spatial inequality of disaster impact. Considering the input data complexity and various formats, this work proposes a deep neural network (DNN)-based framework to represent the nonlinear relationship between the social media damage description and straightforward monetary damage. Particularly, the social media data in Winter Storm Uri and damage data collected by FEMA are employed to train, validate, and test such a DNN framework. The content of this paper is organized as follows. Section 2 reviews the related work and how this work is related to the current literature. Section 3 demonstrates the methodology, including the details of data collection, DNN architecture design, training, and testing arrangement. Following that, Section 4 shows the results analysis and the estimation products. Lastly, Section 5 provides the conclusion and lists the future research directions.

2. Related Work

There are many aspects in harnessing social media to assist disaster response and recovery [19,20]. One of the methods is the sentiment sensing based on natural language processing (NLP) technique. It extracts people’s reactions and situation awareness by classifying the text data of social media into positive, negative, or neutral categories [21]. Another aspect is information fission and coordination. For instance, a research team from Washington created Twitris to classify emergency-related topics and visualize the results on a Geographic Information System (GIS) map for the general public [22]. Mobility pattern can also be measured from social media. For instance, georeferenced tweets were utilized to quantify and predict New York City citizens’ movements before and after Hurricane Sandy [23,24].

Besides the above-mentioned various applications, damage estimation is critical to efficient first response and resource delivery, and a wide range of methods have been used to address this problem. For instance, social media was adopted to derive a disaster index related ratio (DIRR) and sentiment in each county in Florida during Hurricane Matthew, and suggested the positive correlation between damage claims and DIRR and sentiment [9]. Other research studied Hurricane Sandy by utilizing a multiscale analysis of Twitter activity before, during, and after the event, concluding that negative sentiment correlates with per-capita damage [25]. Even though correlations have limited applications, they show that it is possible to inform damage from social big data.

Other than measuring such correlations, computer vision techniques have been used to measure the severity of the impact. One research team constructed convolutional neural network (CNN) models to classify hurricane and earthquake damage images from social media into three categories (none, mild, severe) with 76% accuracy [26]. One more recent work fused CNN and natural language processing (NLP) techniques to generate captions for the social media images that enable victim and building damage identification [27]. With proper aggregation and analysis, these images can provide the detailed severity description to the users, including government, volunteers, and first responders. However, this method requires images taken from cameras and angles similar to the training dataset, which are not always available on social media. Despite the works of correlation analysis, GIS mapping, and computer vision classification, there is limited work on the end-to-end monetary loss estimation from social media, especially at the household level. A deep neural network (DNN), which has many layers of neurons, hence deep, is good at modeling complex relationships between a wide range of input and output [28]. For example, a supervised DNN with enough data can model chemical and physical processes with high speed [29], recognize images via CNN architectures [30], and predict stock market returns based on historical data [31]. The reason behind this is that the high-dimensional neural connections and weights can be optimized by many iterations based on the training data. Therefore, this work attempts to utilize a DNN to map the social media data (texts, images, and videos) to the straightforward monetary loss.

3. Methodology

3.1. DNN Structure

Equation (1) shows the high-level concept of the DNN design. In this equation, z indicates zip code and DNN represents the DNN function. Variables Text, Image, and Video are all the vectorized texts, images, and videos from social media located within that zip code area z. On the output side of the DNN function is the numerical description of the impact, such as Damage and Injury, in each zip code area. It is worth noting that the data categories on both input and output sides of the equation are expandable, e.g., the amount of displaced people can append to the output, as long as the data can be vectorized (converted to numerical values). With this end-to-end structure, the DNN model will produce the numerical estimates, such that the social media usage bias mentioned in Section 1 will be modeled in the DNN weights. Indeed, there is bias in the intermediate features; however, the accurate estimation on the output end provides valuable information regardless.

The reason for selecting zip code as the model building granularity is twofold. First, each zip code area covers similar population in Texas, as shown in Figure 1, compared to the larger population across counties. According to this histogram, most zip code offices serve fewer than 60,000 people, whereas the most populated county in Texas is Harris County, with 4.7 million residents. The second is that FEMA publishes household-level damage data with zip code and county information but without precise location (latitude and longitude) for privacy protection. Therefore, the zip code is selected to align the input and output resolution. In other words, this DNN is designed to sense the disaster reaction and records in each zip code, based on the underling similar resiliency and behavior commonalities in close communities.

$$DNN\left[\begin{array}{c}Tex{t}_z\\ Imag{e}_z\\ Vide{o}_z\\ ...\end{array}\right]=\left[\begin{array}{c}Damag{e}_z\\ Injur{y}_z\end{array}\right]...$$

(1)

Figure 2 shows the details of the DNN architecture in this research with the input (green box states “text”) on the left and output (blue box states “damage data”) on the right. This work aims to test the plausibility of such a DNN design; hence, only the text data are used, rather than fully integrating multiple data formats such as images and videos. Nevertheless, image and video data can also be vectorized by using CNN, long short-term memory (LSTM), and other techniques such that they can be combined with text data as input. On the input side, all the text data in zip code (z) will be stacked together and vectorized to a 1024-dimensional vector using HashingVectorizer [32]. In the network in between, i.e., hidden layers, there are 5 fully connected layers marked with yellow color. The number of neurons of each hidden layer is 512, 256, 128, 64, and 32, respectively. Lastly, the output layer is a 7-dimensional vector representing the average economic loss in 7 categories defined by FEMA. The input and output neurons are also fully connected with the hidden layers. At the bottom of Figure 2, the dimensions of input, each hidden layer, and output are marked in solid boxes with the corresponding color.

Between each two connected neurons, activation function rectified linear units (ReLU) is used to add non-linearity [33]. The L1 loss function, as shown in Equation (2), is used in the training process. In this equation, z indicates the zip code and c represents the FEMA damage category, $damag{e}_{true}$ is the ground truth (FEMA) average household damage in USD, and $damag{e}_{predict}$ is the DNN predicted damage data. The prepared dataset is split into three parts: training (80%), validation (10%), and testing (10%). While training, the DNN weights are optimized by using the stochastic gradient descent technique [34] to minimized the L1 loss so that the DNN predictions are as close as possible to the FEMA damage data. The initial learning rate is set to be 0.001 and decreases by one magnitude if no L1 loss decrease occurs in 3 iterations. To avoid overfitting, the validation loss is monitored and the training process is terminated if the validation loss does not decrease within 10 iterations [35]. Lastly, the testing portion is used to report the DNN performance.

$$L1loss=\sum _{i=1}^z\sum _{j=1}^c\left|damag{e}_{true}-damag{e}_{prediction}\right|$$

(2)

3.2. Social Media Data and Keywords Calculation

Previous studies have collected data from a variety of social media, such as Facebook, Twitter, and Flickr. However, some platforms limit data access due to privacy concerns, e.g., Facebook. This research uses Twitter because it allows users to share all types of data (e.g., texts, images, videos, polls, links, and hashtags) with geotags, which is critical to spatial damage analysis and response. Most importantly, Twitter recently launched an “academic research product track” providing access to 10 million records per month from its historical data repository [36], which was not available before. Therefore, the Twitter application programming interface (API), which is a keywords based system, is used to collect tweets during Uri in Texas.

There are broadly three types of social media noise: constant background topics (e.g., daily traffic), overlapping event (e.g., Valentine’s day and Uri), and random signals (e.g., advertisement robots) [37]. In order to only mine the winter-storm-related text data, this work introduces a keywords calculation mechanism to generate closely related keywords for the Twitter API system. In detail, 1% (2 million) of all the global tweets in February 2021 is collected from an online library named Internet Archive (accessed 1 November 2021 from https://archive.org/details/twitterstream). Next, all the tweets with location in Texas are divided into three parts by Uri’s timeframe of 11–21 February 2021 (FEMA definition): February before (FB), February during (FD), and February after (FA), meaning the tweets before, during, and after Uri, respectively. In addition, a week’s worth of current data (from 1 October) is collected to form a subset of October current (OC). The purpose of such data arrangement is to automatically calculate Uri-related keywords using Equation (3), followed by using these keywords to search as many data as possible via Twitter API.

$$keywords=(FD+FA)-(OC+FB)$$

(3)

In this equation, FB, FD, FA, and OC are lists of keywords extracted from FB, FD, FA, and OC tweets using Rake [38]. The term keywords represents the desired keywords for Twitter API. The reasoning of Equation (3) is as follows.

Table 1. Similarity (%) matrix of keywords of FB, FD, FA, and OC.

	FB	FD	FA	OC
FB	100	52.22	65.55	22.22
FD	52.22	100	56.66	15.55
FA	65.55	56.66	100	21.11
OC	22.22	15.55	21.11	100

shows the cosine similarities [39] comparison of these four portions. According to this table, OC shares the least similarity among them due to its different season, and appears to be closer to FB and FA than FD. The reason could be that OC, FB, and FA are regular times, whereas FD is undergoing a storm impact. Looking at February only, FD appears to be closer to FA than FB. This implies that the impact of Uri continues in the aftermath FA which is different from regular time FB. Therefore, in Equation (3), the operator “+” combines the keywords during and after Uri (FD + FA) and regular time keywords (OC + FB). Then, operator “−” subtracts the two and removes regular time keywords, leaving only Uri-related keywords. Given the maximum API limit of 1000 characters (roughly 92 words excluding necessary operators such as location), the top 92 keywords were used to mine the Twitter repository without retweets. The full list is below:

“sanantonio OR career OR opening OR warm OR jobs OR park OR interesting OR alerts OR cold OR event OR stones OR measured OR mexico OR heavy OR due OR group OR mobile OR manager OR security OR fun OR reports OR media OR details OR sales OR snow OR turn OR recommend OR facebook OR view OR video OR apply OR places OR blvd OR landmark OR utc OR stepping OR shreveport OR exit OR santa OR rock OR discover OR valentine OR cleburne OR tree OR dollar OR inch OR odessa OR link OR ave OR public OR atxtraffic OR nurse OR hiring OR place OR titles OR chase OR technician OR san OR blocked OR open OR case OR read OR oklahoma OR university OR gainesville OR hear OR pkwy OR bank OR engineering OR follow OR center OR west OR traffic OR ice OR stay OR antonio OR earthquake OR wsw OR cst OR weather OR latest OR wnw OR service OR click OR plano OR left OR bio OR store OR vday OR okctraffic OR created OR fort OR round”.

The operator “OR” in between each word is to define the search logic, i.e., tweets containing any of the keywords are defined as relevant. This list contains unique keywords to winter storm such as “warm”, “cold”, and “snow”. Nevertheless, other seasonal keywords, such as “valentine” are also in this list. According to the latest Twitter features, users can geotag a tweet with place (polygon with multiple points) or geoinformation (one point with latitude and longitude). The place polygon can be as big as Texas or as small as a park. While mining, the geotag filter is “place:Texas” which defines a rectangle with the most north–west and east–south coordinates for content filtering. Due to this rectangle covering some parts of Mexico, Oklahoma, and other neighboring regions, keywords such as “oklahoma” and “mexico” are also in the list.

Mining with the above-mentioned 92 keywords resulted in a dataset named Uri with tweet text strings, image, and video attachment URLs if available, user ID, time stamp, geoinformation if available, and place ID. Altogether, 16,028 tweets in FD and FA periods were mined within Texas, with 4934 with attachment URLs (images or videos). However, only 1143 of them have latitude–longitude coordinates, which is consistent with the fact that less than 1% of tweets tag their pinpoint locations [40]. All these coordinates were used to compute the zip code for each tweet in the dataset Uri.

3.3. Damage Statistics from FEMA

The household damage was collected via OPENFEMA API [41], which covers all historical public assistance (PA) and individual assistance (IA) approved by FEMA in all national declared disasters, including Uri. Each household has a record with up to 96 categories of information, including income, rent/own status, occupancy number/age, structure type, insurance, and others. Since the focus of this paper is the household-level damage, only the IA data are used.

Table 2. Selected FEMA IA categories.

Category	Description	Source
ihpAmount	Total individual and households program (IHP) amount awarded in USD for eligible applicants.	IA owner, IA renter, IHP
haAmount	Amount awarded for housing assistance (HA) in USD from IHP.	IHP
onaAmount	Amount awarded in USD for other needs assistance (ONA) from IHP.	IHP
rpfvl	Real property damage amount.	IHP, IHP large
ppfvl	Value of disaster-caused damage to personal property components, including appliances and furniture.	IHP, IHP large
rentalAssistanceAmount	Amount of rental assistance awarded in USD.	IA renter, IHP
repairAmount	Amount of repair assistance awarded in USD.	IA owner, IHP

displays the monetary economic loss categories with their description and source: individual and households program (IHP) amount (ihpAmont), housing assistance (HA) amount (haAmount), other needs assistance (ONA) amount (onaAmount), real property damage amount (rpfvl), personal property (ppfvl), rental assistance amount (rentalAssistanceAmount), and repair amount (repairAmount). The data in each category are all aggregated and averaged at the zip code level and paired with social media data.

The completed dataset Uri has social media data from 550 different zip codes and FEMA damage data from 1587 zip codes. Considering that Texas has 1930 zip codes in total, FEMA does not cover all the zip codes, especially rural regions (visualized in Section 4). Altogether, the dataset Uri has 338 paired data points with both social media and damage data as input and output for the DNN building. According to the methodology, 270 (80%) data points were used as training and the rest 68 (20%) were for validation and testing. There are 212 zip codes that have social media relevant to the winter storm but no FEMA damage reports. The fully trained DNN will be used to estimate the missing damage statistics, hence it is important to assist the communities at a disadvantage [42].

4. Analysis and Results

Based on the hyperparameters defined in Section 3.1, the DNN model is trained on the training portion and terminated at roughly 9000 iterations. The fully trained model is then tested on the testing portion to compare the prediction with the ground truth. Figure 3 shows the comparison for each testing zip code, where the X-axis indicates the prediction average loss in USD, and the Y-axis represents the damage measured by FEMA. In the 100% accurate scenario, all the testing points of all categories (colors) should fall on the 45 degree line. By definition, the points in the upper left section are the underestimation cases and the points in the lower right section are the overestimation cases. According to this figure, underestimation is more common than overestimation, especially at higher values, e.g., ihpAmount.

The accuracy for each category c is calculated using Equation (4) below. Here, z represents the zip code in the testing portion, $damag{e}_{true}$ is the ground truth FEMA damage, and $damag{e}_{predict}$ means the DNN-predicted damage data. We choose this metric since it is equivalent to averaging relative error over each estimate, but is weighted by the ground truth value. According to this figure, the best-performing categories are ihpAmont, haAmount, and rentalAssistanceAmount, achieving accuracy of 68.43%, 68.39%, and 70.07%, respectively. Moreover, the smaller values, e.g., ppfvl and onaAmount, often are associated with lower precision (51.89% and 34.35%) compared to others.

$$\begin{array}{c}\hfill accurac{y}_c=1-\sum _{i=1}^z\frac{\left|damag{e}_{true}-damag{e}_{prediction}\right|}{damag{e}_{true}}\end{array}$$

(4)

As mentioned in Section 3.3, FEMA did not survey 212 zip codes for the storm damage. However, this does not mean there is no damage in those areas. The fully trained DNN model can be used to estimate the average damage amount for them in all seven categories based on real-time social media data. To visualize such capability on a GIS map, the ihpAmount is selected because it has a high value, thus clear variations. Figure 4 illustrates the ihpAmount from FEMA and the DNN model. In this map, the gray area represents that there are no data from both social media and FEMA. The 338 paired data points in the dataset Uri, i.e., zip codes with both FEMA and social media data, are marked with green color representing the FEMA evaluated amount in USD. The intensity of the color shows the value variation in each legend. There are zip codes with only social media but no FEMA evaluation, and the DNN can estimate the damage for these neglected regions. The DNN-estimated ihpAmount is color-coded with red, indicating the USD values in five bins: 0–105, 105–208, 209–319, 319–429, and 429–533. Similarly, the zip codes that only have FEMA damage data are marked with blue and are visualized with different color intensity.

From this map, it is observed that many zip codes in rural West Texas do not have FEMA survey data, whereas metropolitan areas such as Houston and Dallas are fully covered (blue and green). In comparison, the DNN-estimated damage has less variation (0–522) than the FEMA damage data (0–10,070). Still, some zip codes in rural West Texas miss both social media and FEMA data. This can be solved by localizing more users there even they do not share geoinformation. Densely populated areas, i.e., small-sized zip codes, often have less damage than large-sized zip codes in both the DNN and FEMA damage. It is also observed that several zip codes in Oklahoma are reported by FEMA and are hence mapped in this figure.

In order to evaluate the damage prediction accuracy across geographies, Figure 5 displays the ihpAmount prediction error for each paired zip code, i.e., green in Figure 4. The error percentage is calculated by Equation (5) for each zip code, where $damag{e}_{true}$ is the FEMA ihpAmount and $damag{e}_{predict}$ means the DNN predicted value. According to this definition, the lower the error (coded with lighter color), the closer the prediction is to the FEMA survey, and the darker the color, the less accurate for the DNN estimation. The color intensity is divided into 10 equal bins by data size, with the remaining marked in light blue. It is worth noting that this map only shows the overlapping (FEMA survey and DNN estimation) ihpAmount to show the accuracy as an example; the other categories can be analyzed the same way. In this map, 7 out of 10 bins have an estimation error below 43%, which is consistent with the average ihpAmount 68.42% accuracy in Figure 3. The more accurate estimations appear close to the cities (Houston, Dallas, and San Antonio) compared to rural regions. The map distribution also suggests more data collection in the centers of Houston and Dallas areas, as they are voids. Altogether, the DNN only trained on text data is capable of predicting average damage with up to 70% accuracy and is able to estimate damage for the communities that are often neglected.

$$\begin{array}{c}\hfill error=\frac{\left|damag{e}_{true}-damag{e}_{prediction}\right|}{damag{e}_{true}}\end{array}$$

(5)

5. Conclusions and Discussion

This work demonstrated a prototype that intakes georeferenced real-time social media data (texts, images, and videos) and estimates the monetary damage statistics using DNN techniques. The output estimation could be visualized and mapped for end users including government, non-governmental organizations (NGOs), first responders, news media, and the general public. To test the plausibility of such DNN model, a keyword calculation technique was developed to mine the text data relevant to Winter Storm Uri on Twitter. The damage statistics of seven different categories were collected from FEMA individual assistance reports and paired with the social media data on the zip code level to form the dataset Uri. A fully connected DNN structure was designed and trained, validated, and tested on the dataset Uri to predict the average monetary loss in USD based on the georeferenced text data. The fully trained DNN model achieved up to 70% precision when tested on the testing portion, indicating successful damage estimation. Moreover, the fully trained DNN could help in estimating the damage where FEMA survey does not cover, especially fpr rural and low-income communities who are often at a disadvantage.

Although only 1143 tweets from 550 zip codes were used in this study, the fully trained DNN was able to achieve a 70% accuracy, which shows the potential of the proposed method. The total collected data size is 16,028 tweets (with images), with a great portion of them not having location information. Based on the common understanding of DNN research, the more training data there are, the more accurate the model becomes. Hence, the next step is to develop tweet localization techniques to enlarge the dataset Uri. This could be achieved by using the user’s previous location, social network region, or other information to derive the current geoinformation. However, this is beyond the scope of this research. Moreover, the information-rich image and video (images and audio) data can be combined with the current DNN, which is another way to increase the data size. This work is set to predict zip-code-level average damage due to the location information availability. Future work of downscaling, i.e., from zip-code-level average onto household level, could improve the granularity of this technique and, hence, greatly assist general public risk awareness, resource distribution, and volunteer arrangement. Finally, we seek to generalize our approach to investigate the extent to which our approach, and even a specific regional model, is applicable to comparable events occurring in other regions.