Development after Displacement: Evaluating the Utility of OpenStreetMap Data for Monitoring Sustainable Development Goal Progress in Refugee Settlements

Abstract

In 2015, 193 countries declared their commitment to “leave no one behind” in pursuit of 17 Sustainable Development Goals (SDGs). However, the world’s refugees have been routinely excluded from national censuses and representative surveys, and, as a result, have broadly been overlooked in SDG evaluations. In this study, we examine the potential of OpenStreetMap (OSM) data for monitoring SDG progress in refugee settlements. We collected all available OSM data in 28 refugee and 26 nearby non-refugee settlements in the major refugee-hosting country of Uganda. We created a novel SDG-OSM data model, measured the spatial and temporal coverages of SDG-relevant OSM data across refugee settlements, and compared these results to non-refugee settlements. We found 11 different SDGs represented across 92% (21,950) of OSM data in refugee settlements, compared to 78% (1919 nodes) in non-refugee settlements. However, most data were created three years after refugee arrival, and 81% of OSM data in refugee settlements were never edited, both of which limit the potential for long-term monitoring of SDG progress. In light of our findings, we offer suggestions for improving OSM-driven SDG monitoring in refugee settlements that have relevance for development and humanitarian practitioners and research communities alike.

1. Introduction

In 2015, as part of the United Nations (UN) 2030 Agenda for Sustainable Development, 193 countries declared their commitment to “leave no one behind” in the shared pursuit of 17 Sustainable Development Goals (SDGs; Figure 1), including “No Poverty” (SDG 1), “Gender Equality” (SDG 5), “Climate Action” (SDG 13), and “Peace, Justice, and Strong Institutions” (SDG 16). The Agenda was designed to “ensure that all human beings can fulfil their potential in dignity and equality and in a healthy environment” [1], but the global population of refugees—those forcibly displaced due to violent conflict, political persecution, and oppression—has grown every year since. In 2015, the United Nations High Commissioner for Refugees (UNHCR) documented 16 million refugees who had been granted asylum across international borders [2]; in the five years that followed, an additional five million people (representing a 22% increase) were granted refugee status, with two million requesting asylum in 2019 alone [3]. Refugees commonly experience landlessness, joblessness, marginalization, food insecurity, and a loss of access to common property resources while in exile [4,5,6,7], all of which constrain their potential for sustainable development [8]. The challenges faced by refugees are hardly short-lived. Even though refugee settlements are often thought to be a temporary solution, the average stay in a refugee settlement was 10 years as of 2015 [9], and more than two-thirds of refugees experience what the UNHCR calls a “protracted refugee situation”, in which refugees remain in limbo for years on end, unable to return home but without provision of basic rights and access to economic and social services in their host country [10].

To improve long-term human, economic, and environmental development outcomes for refugee populations, information on SDGs within refugee settlements is essential. The SDGs are most commonly assessed through national censuses and surveys; however, refugee populations are consistently excluded from these data collection approaches as well as global settlement and population datasets [11,12,13,14,15]. Despite making a commitment to “leave no one behind” and extensive humanitarian monitoring conducted in refugee camps, only 26 of 47 refugee-hosting countries identified refugees in their national SDG progress reports in mid-2020 [16]. Given a variety of logistical and political challenges [14] and a persistent humanitarian-development divide [17,18], major refugee hosting countries rarely comment on SDG outcomes for refugees or describe refugees as meriting specific attention [15], and sub-national information on refugee housing, energy, education, transportation, water, and medical care, for example, are rarely provided [14]. There has also been little academic scholarship on monitoring SDG progress in refugee settlements (with exceptions, [19,20]) and most attention has gone toward SDG 7 (Clean and Affordable Energy), e.g., References [21,22,23]. Compounding the lack of visibility of refugee data for SDG assessments, the only mention of refugees in the long list of SDG indicators comes with SDG 10.7.4, as “Proportion of the population who are refugees, by country of origin”, which refers to the refugee country of origin rather than asylum and gives no insight into the development conditions experienced by refugees. Without relevant data and explicit reference to refugees in SDG monitoring, refugee-hosting countries are less likely to develop, pursue, and reach refugee-focused development targets [14].

As the SDGs call for open and inclusive processes and methodologies, OpenStreetMap (OSM) data are a potentially well-suited, though underexplored, avenue for assessing SDG progress at refugee settlements [24,25,26,27]. OSM data are most commonly used for mapping urban infrastructure [28,29,30], offer open-access, global coverage, hyperlocal geo-precision, thematic richness, and temporal depth [31,32,33], and have been used broadly by humanitarian scholars and practitioners [34,35,36,37,38]. We examined the utility of OSM data for monitoring SDG progress in UNHCR refugee settlements in the major refugee-hosting country of Uganda. We collected all available OSM data within 28 refugee settlements and 26 non-refugee settlements and created a novel SDG-OSM data model linking 149 OSM feature tags to 11 SDGs. With these SDG-OSM pairings, we (1) quantified the spatial distribution of SDG-relevant OSM data across and within settlements, (2) measured the chronology of creation and versioning of SDG data, and (3) compared the spatial and temporal coverages of SDG data between refugee and non-refugee settlements. Our findings illustrate both the potential and limitations of using OSM data for SDG monitoring in refugee settlements, we discuss several factors that explain differences between OSM and field survey-based SDG assessments, and we conclude by offering suggestions for improved OSM-driven SDG monitoring in refugee settlements. This study is the first to undertake such an extensive and systematic analysis of OSM data for refugee SDGs assessments with relevance for open mapping, development, and humanitarian practitioner and research communities alike.

2. Study Area

As of writing, Uganda has the fourth largest refugee population in the world, with 1.4 million refugees (nearly 3% of Uganda’s total population) under UNHCR protection [39]. Uganda’s earliest refugees settled in the 1960s, but the population increased almost ten-fold between 2012 and 2017 with the arrival of refugees from South Sudan, the Democratic Republic of the Congo (DRC), and Burundi (Figure 2). Given that the same conflicts that forced refugees to leave their home countries in the first place are often ongoing, many refugees in Uganda do not intend to return to their home country and are likely to stay in Uganda for years to come [40]. Approximately 92% of Uganda’s refugees live in 30 UNHCR-managed settlements in the Northern and Western Regions of Uganda (Figure 3), and the remaining 8% of Uganda’s refugee population lives in the capital city of Kampala [41]. Refugee settlements are typically small, with a median area of 4 km² (range: 0.2 to 790 km²) and a median population of 16,782 people (mean: 47,619, standard deviation (SD): 58,093) as of September 2020 (

Table 1. Characteristics of 28 study refugee settlements in Uganda. Settlement establishment year data based on UNHCR Settlement Factsheets. Primary country of origin and population data (as of September 2020) from data2.unhcr.org/en/documents/details/79424 (accessed on 1 December 2020). Settlement area based on OSM settlement boundary data. DRC: Democratic Republic of the Congo.

Settlement Name	Year Established	Region	Primary Country of Origin	Area (km2)	Population
Agojo	2016	Northern	South Sudan	6.7	7167
Alere	2013	Northern	South Sudan	1.2	6882
Ayilo I	2015	Northern	South Sudan	4.9	26,051
Ayilo II	2014	Northern	South Sudan	2.9	14,623
Baratuku	2013	Northern	South Sudan	2.2	7049
Bidi Bartok	2016	Northern	South Sudan	790.6	232,726
Boroli I	2014	Northern	South Sudan	0.7	10,098
Boroli II	2015	Northern	South Sudan	0.4	5138
Elema	1992	Northern	South Sudan	3.2	991
Imvepi	2017	Northern	South Sudan	96.7	69,192
Kiryandongo	2014	Western	South Sudan	41.7	67,704
Kyaka II	2017	Western	DRC	45.1	123,831
Kyangwali	1960	Western	DRC	96.3	123,025
Maaji I	1997	Northern	South Sudan	0.3	548
Maaji II	2015	Northern	South Sudan	3.3	17,518
Maaji III	2015	Northern	South Sudan	2.6	16,046
Mireyi	1994	Northern	South Sudan	0.2	7067
Mungula I	1996	Northern	South Sudan	1.1	5028
Nakivale	2015	Western	DRC	458.2	133,192
Nyumanzi	2014	Northern	South Sudan	5.2	40,877
Oliji	2013	Northern	South Sudan	1.1	1420
Olua I	2012	Northern	South Sudan	0.5	5359
Olua II	2012	Northern	South Sudan	0.4	4241
Pagirinya	2016	Northern	South Sudan	7.2	36,784
Palabek	2017	Northern	South Sudan	207.1	53,806
Palorinya	2016	Northern	South Sudan	114.7	122,805
Rhino Camp	1980	Northern	South Sudan	490.3	121,171
Rwamwanja	2012	Western	DRC	79.2	72,997

). Settlements are broadly self-contained with housing, transportation infrastructure, food markets, financial services, and educational and healthcare facilities on-site, as well as refugee response offices, which process and register incoming refugees. In addition to the large and recently arrived refugee population, refugee settlement boundaries in Uganda are available in the OSM database, making the country an ideal case study.

In service of the UN Global Compact on Refugees (GCR) signed in December 2018 [42], Uganda’s progressive refugee policies, including a “no camp” policy, recognize the importance of self-sufficiency and freedom of movement for refugees, refugee access to government services, and refugee contributions to broader socioeconomic development [43,44]. For example, each family living in Ugandan refugee settlements is allocated a plot (30 by 30 m) for housing and agricultural cultivation, and refugees in Uganda are permitted to own their own business and be employed throughout the country [41]. Uganda’s national development policies associated with education, health, and water explicitly engage the development needs of refugees [45], however, refugee SDG progress in Uganda still lags far behind the national average [46]. Concerning SDG 1 (No Poverty), national survey data collected in 2018 showed that nearly half (46%) of Uganda’s refugees live in poverty, compared to a 21.4% national poverty rate as of 2016 [47], and that refugees experience twice the rate of poverty as host communities. Concerning SDG 2 (Zero Hunger), 70% of Uganda’s refugees experienced severe food insecurity in 2018 even though 70% of refugees had access to cultivable land. Concerning SDG 6 (Clean Water and Sanitation), drinking water is not distributed equitably across refugee settlements, and over a third of South Sudanese refugees lacked a household latrine. Concerning SDG 7 (Affordable and Clean Energy), 93% of refugee households rely on fuelwood energy for cooking and spend 22% of their income on fuelwood. Concerning SDG 8 (Decent Work and Economic Growth), only 28% of refugees are employed and are paid 35–45% lower than residents of non-refugee communities. These disparities reveal that despite enjoying liberal refugee policies, national SDG progress in Uganda does not reflect the realities faced by refugees living in the 28 UNHCR settlements that are the focus of this study. Using Uganda as a case study for this novel OSM application may provide unique insights into how SDGs can be monitored in a relatively data-rich context with liberal policies, which may then be applied to refugee populations in other more challenging settings.

3. Materials and Methods

3.1. OpenStreetMap Data

OpenStreetMap (openstreetmap.org (accessed on 1 December 2020)) is a georeferenced, crowdsourced product based on data collected in the field and through interpretation of remotely sensed aerial or satellite imagery. In recent years, OSM has become an essential source of geospatial information on infrastructure and place-specific services in historically under- or never-mapped regions [26,28,33,48,49,50]; however, to date, there are few examples of OSM-driven analyses within refugee settlements, e.g., References [51,52]. We downloaded the Uganda-wide OSM dataset through GeoFabrik (download.geofabrik.de/africa/uganda.html (accessed on 1 December 2020)) in August 2019. These data consist of a feature stored as a node, way, or relation. Nodes are zero-dimensional features with latitude and longitude and are either a point feature or form part of a way. Ways are an ordered set of nodes used to form one-dimensional features (such as a path or wall) or two-dimensional features (such as a settlement boundary or park perimeter). Relations are most commonly used to represent a network of related nodes and ways (such as a road network).

In Uganda, the majority of OSM data at refugee settlements were collected by the Humanitarian OpenStreetMap Team (HOT) [53]. Of the 30 UNHCR-managed refugee settlements in Uganda, we identified boundaries in OSM data for 28 refugee settlements (Table 1). Boundary data were not available for Lobule and Oruchinga refugee settlements at the time of data collection. These refugee settlement boundaries (Figure 3 and Figure 4) were established by UNHCR in agreement with the Government of Uganda and added to OSM by HOT. We subset the Uganda OSM dataset by refugee settlement boundaries, identified OSM nodes within refugee settlements, and collected all metadata for these nodes using Overpass Turbo (overpass-turbo.eu (accessed on 1 December 2020)). This yielded 23,818 nodes in refugee settlements representing physical features associated with dwellings, schools, clinics, latrines, etc., with metadata on feature creation date, date of most recent edit, and descriptive tags encoded as key-value pairs.

3.2. GRID3 Non-Refugee Settlement Boundary Data

Since OSM data in Uganda generally do not include boundaries of rural settlements in refugee-hosting regions, we used the Africa-wide GRID3 (Geo-Referenced Infrastructure and Demographic Data for Development) dataset [54,55,56]. GRID3 is a human-validated, open-source geospatial dataset in which settlement boundaries are generated by clustering individual buildings detected in very high-resolution Maxar satellite imagery (Figure 5) [57]. To develop a comparative sample of OSM data at non-refugee settlements, we identified three GRID3 Built-Up Area (BUA) boundaries nearest to each refugee settlement based on centroid-to-centroid distance; after removal of duplicate BUA boundaries, we arrived at 26 unique non-refugee settlements (median area: 11.11 km², mean: 13.78 km², standard deviation: 11.19 km²). As with refugee settlements, we downloaded all OSM data within non-refugee settlement GRID3-based boundaries through Overpass Turbo, amounting to 2465 total OSM features.

3.3. OSM Populated Places Data

We used OSM Populated Places data [58] to assign settlement names to GRID3 settlement boundaries. Of the 26 GRID3 boundaries, seven did not contain an OSM Populated Place feature. For these boundaries, we assigned the name of the nearest OSM Populated Place feature to the GRID3 settlement boundary (median distance: 3.97 km, mean: 4.32 km, standard deviation: 3.02 km). For the 13 GRID3 boundaries that contained more than one Populated Place feature, we assigned the name of the feature with the highest population to the GRID3 settlement boundary. If population data were unavailable, we used the name of the Populated Place feature nearest to the center of the GRID3 settlement boundary.

3.4. Sustainable Development Goals and OpenStreetMap Metadata Data Model

Excluding source and place tags, we identified 3665 unique OSM feature tags (i.e., key-value pairs) within the 28 refugee settlement boundaries. Of these, we identified 149 tags relevant for a total of 11 SDGs (Appendix A

Table A1. SDG-OSM data model. OSM tags are shown as “key—value” pairs.

SDG	OSM Tag
1. No Poverty	amenity—mobile_money_agent
1. No Poverty	amenity—bank
1. No Poverty	amenity—microfinance_bank
1. No Poverty	amenity—mobile_money
1. No Poverty	amenity—microfinance
1. No Poverty	amenity—atm
1. No Poverty	amenity—mobile_money
1. No Poverty	amenity—banking_agent
1. No Poverty	amenity—credit_institution
1. No Poverty	amenity—sacco
1. No Poverty	amenity—bureau_de_change
1. No Poverty	amenity—money_transfer
1. No Poverty	network—mtn_mobile_money
2. Zero Hunger	amenity—marketplace
2. Zero Hunger	amenity—restaurant
2. Zero Hunger	building—farm
2. Zero Hunger	leisure—garden
2. Zero Hunger	land use—farmland
2. Zero Hunger	land use—farm
2. Zero Hunger	place—farm
2. Zero Hunger	shop—beverages
2. Zero Hunger	shop—butcher
2. Zero Hunger	shop—bakery
2. Zero Hunger	shop—food
2. Zero Hunger	shop—greengrocer
2. Zero Hunger	shop—supermarket
3. Good Health and Well-Being	amenity—doctors
3. Good Health and Well-Being	amenity—clinic
3. Good Health and Well-Being	amenity—hospital
3. Good Health and Well-Being	amenity—pharmacy
3. Good Health and Well-Being	amenity—nursing_home
3. Good Health and Well-Being	amenity—health_center
3. Good Health and Well-Being	healthcare—clinic
4. Quality Education	amenity—school
4. Quality Education	amenity—kindergarten
4. Quality Education	amenity—college
4. Quality Education	amenity—university
4. Quality Education	amenity—library
4. Quality Education	amenity—language_school
5. Gender Equality	amenity—childcare
6. Clean Water and Sanitation	amenity—toilet
6. Clean Water and Sanitation	amenity—toilets
6. Clean Water and Sanitation	amenity—water_point
6. Clean Water and Sanitation	amenity—private_toilet
6. Clean Water and Sanitation	amenity—drinking_water
6. Clean Water and Sanitation	amenity—public_bath
6. Clean Water and Sanitation	man_made—water_tank
6. Clean Water and Sanitation	man_made—borehole
6. Clean Water and Sanitation	man_made—water_well
6. Clean Water and Sanitation	amenity:drinking_water—drinking water
6. Clean Water and Sanitation	pump—hand_pump
6. Clean Water and Sanitation	pump—manual
6. Clean Water and Sanitation	pump—powered
6. Clean Water and Sanitation	waste—sanitary waste
6. Clean Water and Sanitation	waste—urinal
8. Decent Work and Economic Growth	amenity—internet_cafe
8. Decent Work and Economic Growth	amenity—office
8. Decent Work and Economic Growth	amenity—Market
8. Decent Work and Economic Growth	building—commercial
8. Decent Work and Economic Growth	shop—convenience
8. Decent Work and Economic Growth	shop—kiosk
8. Decent Work and Economic Growth	shop—hairdresser
8. Decent Work and Economic Growth	shop—shoes
8. Decent Work and Economic Growth	shop—houseware
8. Decent Work and Economic Growth	shop—furniture
8. Decent Work and Economic Growth	shop—electronics
8. Decent Work and Economic Growth	shop—cosmetics
8. Decent Work and Economic Growth	shop—optician
8. Decent Work and Economic Growth	shop—clothes
8. Decent Work and Economic Growth	shop—books
8. Decent Work and Economic Growth	shop—stationary
8. Decent Work and Economic Growth	shop—beauty
8. Decent Work and Economic Growth	shop—hardware
8. Decent Work and Economic Growth	shop—car_parts
8. Decent Work and Economic Growth	shop—car_repair
8. Decent Work and Economic Growth	shop—motorcycle
8. Decent Work and Economic Growth	shop—bicycle
8. Decent Work and Economic Growth	shop—chemist
8. Decent Work and Economic Growth	shop—alcohol
8. Decent Work and Economic Growth	shop—mobile_phone
8. Decent Work and Economic Growth	shop—art
8. Decent Work and Economic Growth	shop—copyshop
8. Decent Work and Economic Growth	shop—charcoal
8. Decent Work and Economic Growth	shop—tailor
9. Industry, Innovation, and Infrastructure	amenity—fuel
9. Industry, Innovation, and Infrastructure	amenity—car_sharing
9. Industry, Innovation, and Infrastructure	amenity—bicycle_parking
9. Industry, Innovation, and Infrastructure	amenity—bus_station
9. Industry, Innovation, and Infrastructure	amenity—parking
9. Industry, Innovation, and Infrastructure	amenity—bicycle_repair_station
9. Industry, Innovation, and Infrastructure	bicycle—yes
9. Industry, Innovation, and Infrastructure	construction—subway
9. Industry, Innovation, and Infrastructure	construction—tram_stop
9. Industry, Innovation, and Infrastructure	generator:source—power_grid
9. Industry, Innovation, and Infrastructure	highway—services
9. Industry, Innovation, and Infrastructure	highway—traffic_sign
9. Industry, Innovation, and Infrastructure	highway—turning_circle
9. Industry, Innovation, and Infrastructure	highway—traffic_mirror
9. Industry, Innovation, and Infrastructure	highway—street_lamp
9. Industry, Innovation, and Infrastructure	highway—give_way
9. Industry, Innovation, and Infrastructure	highway—crossing
9. Industry, Innovation, and Infrastructure	power—tower
9. Industry, Innovation, and Infrastructure	power—generator
9. Industry, Innovation, and Infrastructure	power—pole
9. Industry, Innovation, and Infrastructure	tower:type—lighting
9. Industry, Innovation, and Infrastructure	tower:type—communication
9. Industry, Innovation, and Infrastructure	tunnel—culvert
9. Industry, Innovation, and Infrastructure	waterway—dam
11. Sustainable Cities and Communities	amenity—waste_basket
11. Sustainable Cities and Communities	amenity—waste_dump_site
11. Sustainable Cities and Communities	amenity—waste_transfer_station
11. Sustainable Cities and Communities	amenity—waste_disposal
11. Sustainable Cities and Communities	amenity—recycling
11. Sustainable Cities and Communities	amenity—recycling_type
11. Sustainable Cities and Communities	amenity—waste_basket
11. Sustainable Cities and Communities	amenity—shelter
11. Sustainable Cities and Communities	amenity—community_centre
11. Sustainable Cities and Communities	amenity—post_office
11. Sustainable Cities and Communities	amenity—social_facility
11. Sustainable Cities and Communities	amenity—outreach
11. Sustainable Cities and Communities	leisure—sport_centre
11. Sustainable Cities and Communities	leisure—playground
11. Sustainable Cities and Communities	leisure—swimming_pool
11. Sustainable Cities and Communities	leisure—park
11. Sustainable Cities and Communities	social_facility—reception_centre
11. Sustainable Cities and Communities	social_facility—transit_centre
11. Sustainable Cities and Communities	social_facility—distribution
11. Sustainable Cities and Communities	social_facility—distribution_centre
11. Sustainable Cities and Communities	social_facility—shelter
11. Sustainable Cities and Communities	social_facility—refugee_camp
11. Sustainable Cities and Communities	social_facility—reception_centre__refugee
11. Sustainable Cities and Communities	social_facility—outreach
11. Sustainable Cities and Communities	land use—dumpsite
11. Sustainable Cities and Communities	land use—landfill
11. Sustainable Cities and Communities	waste—all_waste
11. Sustainable Cities and Communities	waste—domestic_waste
11. Sustainable Cities and Communities	waste—rubbish
11. Sustainable Cities and Communities	waste—waste disposal
15. Life on Land	access—forestry
15. Life on Land	land use—forest
15. Life on Land	natural—tree
15. Life on Land	water—pond
15. Life on Land	water—lake
15. Life on Land	water—intermittent
15. Life on Land	water—reservoir
15. Life on Land	water—river
16. Peace, Justice, and Strong Institutions	amenity—embassy
16. Peace, Justice, and Strong Institutions	amenity—police
16. Peace, Justice, and Strong Institutions	amenity—post_office

). We considered using the Open Mapping for the SDGs data model (sdgs.hotosm.org (accessed on 1 December 2020)) but since so many SDGs are not associated with OSM tags, we needed our own data model. The richness of OSM data meant that individual features often contained multiple tags relevant for different SDGs. In cases where a feature could be associated with more than one SDG, we assigned the feature to the most relevant SDG. Considering the example of Vaida Medical Center in Pagirinya (

Table 2. Example of three Sustainable Development Goals (SDGs) and associated OSM tags (shown as “key—value” pairs) at Vaida Medical Center in Pagirinya Refugee Settlement in northern Uganda (OSM Node ID: 5789318243).

SDG 3	SDG 6	SDG 11
amenity—doctors	drinking_water—yes	location—permanent
name—Vaida Medical Center	toilets—yes	operational_status—operational
source:version—Health;v3.0.2	toilets:access—staff, patients	waste_disposal—none

) with tags associated with SDGs 3, 6, and 11, this feature was discretely linked to SDG 3 (Good Health and Well-Being) since healthcare was the essential service of the clinic. Despite developing a liberal approach to linking SDGs and OSM data, there were no OSM data in refugee settlements that we could associate with six SDGs: SDG 7 (Affordable and Clean Energy), SDG 10 (Reduced Inequalities), SDG 12 (Responsible Consumption and Production), SDG 13 (Climate Action), SDG 14 (Life Below Water), and SDG 17 (Partnerships for the Goals). The absence of these SDGs in our data model does not mean that there is no on-the-ground progress towards these SDGs, only that there were not any OSM data associated with these SDGs.

3.5. Objective 1: Quantify the Spatial Coverage of SDG-Relevant OSM Data within Refugee Settlements

We measured the spatial coverage of SDG-relevant OSM data in three ways. First, for each SDG, we measured the total count of SDG data across all study refugee settlements and identified SDGs with the greatest and least overall representation. We also calculated the percentage of settlements with at least one relevant feature for each SDG to characterize the reach of SDG coverage across refugee settlements. Second, for each refugee settlement, we measured the distribution of OSM data by SDG. We examined differences in diversity of SDG representation between settlements and considered whether there were any notable differences in SDG representation between Northern and Western Ugandan settlements. Third, we qualitatively assessed the relative location of SDG-relevant OSM features in individual settlements with respect to road networks and the settlement boundary and visually interpreted the pattern of SDG features as being spatially clustered or diffuse.

3.6. Objective 2: Quantify the Temporality of SDG-Relevant OSM Data Creation and Versioning within Refugee Settlements

SDG monitoring at refugee settlements benefits from having (1) OSM features created soon after settlement establishment to serve as a baseline, (2) routine feature versioning, and (3) recent feature creation to capture the most up-to-date SDG status. For all SDG-relevant OSM data within refugee settlements, we identified the date of feature creation, which we refer to as the Version 1 (V1) date, the date and number of the most recent (i.e., terminal) version, Version n (Vn), and the total number of SDG-relevant OSM features for all possible version numbers. All features have at least one version, and any subsequent edit(s) to a feature results in a version number that is greater than one; for example, a feature with a version number of three (V3) means that the feature was edited two times after creation. We measured the total duration between initial (V1) and terminal (Vn) feature versions, built timelines of SDG data creation and terminal versioning across all refugee settlements, and identified key months of data creation and versioning. We also compared the timing and rate of feature creation, the distribution of the number of versions, and the total duration from V1 to Vn across SDGs. Note that while features such as a latrine, school, or electrical lighting may be permanent without need for versions beyond V1, not all instances of feature versioning, such as an updated spelling of the feature name, are relevant for SDG monitoring. Therefore, we could only consider the presence or absence of versioning without concern for the intent or rationale of the version update.

3.7. Objective 3: Compare OSM-Based SDG Representation at Refugee and Non-Refugee Settlements

To contextualize refugee-centric findings from Objectives 1 and 2 in the broader geographic region, the spatial and temporal coverages of OSM data were evaluated at nearby non-refugee settlements in Northern and Western Uganda. As described above, 26 unique non-refugee settlements nearest to refugee settlements were identified (median centroid-to-centroid distance: 20.8 km, mean: 23.8 km, standard deviation: 13.3 km), and non-refugee settlement boundaries were based on GRID3 BUA (Built-Up Area) data. We conducted comparable spatial and temporal analyses of SDG-relevant OSM data within non-refugee settlements and examined differences in spatial and temporal coverages of OSM data between refugee and non-refugee settlements.

4. Results

4.1. Objective 1: Quantify the Spatial Coverage of SDG-Relevant OSM Data within Refugee Settlements

Approximately 92% (21,950 out of 23,818) of OSM data across 28 refugee settlements in Uganda is related to at least one SDG. Of these data, SDG 6 (Clean Water and Sanitation) is by far the most dominant SDG, with representation across 78% of OSM data, and usually marks the presence of a toilet, latrine, or drinking water feature (Figure 6a). OSM data on SDG 6 are in 27 of 28 settlements (Figure 6b), while five other SDGs (2–4, 8, 11) are represented at over half of the settlements. Unexpectedly, SDG 1 (No Poverty) was not well represented (100 features, 36% settlements) despite the recognized importance of eliminating poverty for achieving sustainable development in refugee settlements [16]. Similarly, SDG 5 (Gender Equality) and SDG 16 (Peace, Justice, and Strong Institutions) were both represented at fewer than 20% of refugee settlements, and OSM data on SDGs 7, 10, 12–14, and 17 were wholly absent, as mentioned above.

The distribution of SDG-relevant OSM data varied across refugee settlements (Figure 7). OSM data on SDG 6 dominates across settlements, except for Elema and Nakivale, which have as much or more data on SDG 8 (Decent Work and Economic Growth), and Rwamwanja, which is mainly covered by SDG 4 (Quality Education). A median of six SDGs (mean: 6, standard deviation: 3) was represented per settlement, though some settlements had data on as few as two SDGs, while one settlement (Imvepi) had 11 SDGs represented across its 609 features. The greatest concentration—73% (16,114 features)—of all SDG data in refugee settlements was found in Bidi Bidi, which is the largest refugee settlement in Uganda (791 km²), with nearly twice the population (232,726 as of 2019) of other refugee settlements. There is no apparent difference in the amount or diversity of SDG representation between Northern and Western Ugandan refugee settlements.

Considering the geographic distribution of SDG data within refugee settlements, data tend to be located near roadways and are generally dispersed throughout a settlement’s extent (Figure 8; Appendix A Figure A1). SDG data are highly clustered, with a median nearest neighbor distance of 45 m. Some settlements, such as Nakivale and Bidi Bidi, are even more clustered, with median nearest neighbor distances of 11 and 17 m respectively, while more diffuse distributions are evident in Pagirinya (129 m) and Palabek (469 m). SDG 6 features tend to be the most dispersed within study settlements, while features for SDG 3, 6, 8, 9, and 11 tend to be clustered together, especially in Ayilo I, Boroli I, and Nyumanzi (see Appendix A Figure A1).

4.2. Objective 2: Quantify the Temporality of SDG-Relevant OSM Data Creation and Versioning within Refugee Settlements

A total of 21,950 SDG-relevant OSM features in Ugandan refugee settlements were created over a five-year period from August 2014 through August 2019 (Figure 9a). In the single month of May 2018, 64% (13,983) of all data were created, almost all of which were within Bidi Bidi and associated with SDG 6—only 3086 features (14%) were created before this month. The general absence of SDG-relevant OSM data in 2014–2017 means that a baseline OSM-driven SDG assessment at refugee settlements can only begin three years or more after most settlements were established and populated. However, data were consistently created from November 2017 through March 2019. An additional 1306 (6%) SDG-relevant OSM features were created in October 2018, mainly between SDG 6 in Bidi Bidi and SDG 8 (Decent Work and Economic Growth) in Kyangwali. Another 1183 (5%) features were created in February 2019, primarily on SDG 8 in Nakivale. Only 33 new features (less than 0.2%) were created from April to August 2019, effectively yielding a five-month gap with no new SDG data in refugee settlements.

Eighty-one percent (17,838) of SDG data were never edited and thus remained Version 1 (i.e., V1) data as of August 2019, and 80% (14,537) of these V1 data correspond to SDG 6 (Clean Water and Sanitation). The remaining 19% (4112) were edited at least once since their creation, resulting in a total of 11,879 feature edits over the five-year study period. Several months of punctuated V1 data creation also saw terminal versioning of Vn data (Figure 9b). For example, February 2016 saw an early co-occurrence between V1 creation and terminal versioning; that is, as new data were collected, just-created data were edited. Terminal versioning mainly occurred between February 2018 and March 2019, with only 3% (122) of Vn features being edited after the March 2019 conclusion of substantial V1 data creation. The lack of recent Vn data is compounded by the late start to V1 data creation and restricts opportunities for OSM-driven monitoring of SDG progress in refugee settlements.

OSM data for many SDGs (e.g., SDGs 1–4) were created gradually over several phases, though some SDGs (e.g., SDGs 5, 6, 9) had half of their data created within acute periods of 1–2 months (Figure 10). The rate of data accumulation bears little relationship to the total amount of data created, as the SDGs with the most (SDG 6) and least (SDG 5) data share similar profiles of an initial rapid and then gradual creation of data. Rapidly created OSM data during a brief period (e.g., SDGs 5, 6, 9) serves more as an SDG status snapshot with less value for monitoring development compared to data that are created over a protracted period (e.g., SDG 4).

OSM data on SDG 5 (Gender Equality) were the most frequently edited, with a median of six versions per feature compared to the typical median terminal version of two or three (Figure 11a). A small minority of OSM data on SDG 3 (Good Health and Well-Being), SDG 4 (Quality Education), and SDG 6 (Clean Water and Sanitation) were edited nine or more times, yielding at least Version 10 data. SDG 5 data were also edited over the longest period of time, with a median V1–Vn duration of 272 days (Figure 11b). SDG 9 (Industry, Innovation, and Infrastructure) data were also highly versioned over a long period of time (median duration: 242 days). Considering both the high number of versions and the long duration between feature creation and terminal versioning, SDG 5 data are the most appropriate for longer-term monitoring.

4.3. Objective 3: Compare OSM-Based SDG Representation at Refugee and Non-Refugee Settlements

Seventy-eight percent (1919) of the 2465 OSM features within 26 non-refugee settlements were relevant for SDG monitoring based on the SDG-OSM data model developed for this study (Appendix A Table A1). This count is less than one-tenth of the 21,950 SDG-relevant features in nearby refugee settlements. Since 78% of OSM data at non-refugee settlements could be associated with specific SDGs, the SDG-OSM is certainly far from irrelevant non-refugee settlements, even though the relationships were based on refugee settlement OSM data. Instead, it seems that refugee settlements have much more OSM data than nearby non-refugee settlements, likely due to targeted HOT field campaigns to collect data in refugee settlements [53].

The distributions of SDGs by total number of features (Figure 12a) and representation across non-refugee settlements (Figure 12b) are markedly different from refugee settlement results. In non-refugee settlements, SDG 4 (Quality Education) and SDG 15 (Life on Land) have the most overall representation but ranked fifth and fourth respectively, in total count across refugee settlements (Figure 6a). SDG 6 (Clean Water and Sanitation) is well-represented in refugee and non-refugee settlements and SDG 16 (Peace, Justice, and Strong Institutions) is poorly represented in both settings. Of note, SDG 5 (Gender Equality) is not represented in non-refugee settlements and has minimal representation in refugee settlements. SDG 4 is depicted across 96% of non-refugee settlements, but no other SDG is represented at more than half of non-refugee settlements (Figure 12b). Refugee settlements, by contrast, have six different SDGs represented across 40% individual refugee settlements, yielding a more diverse depiction of development settlement by settlement. Across non-refugee settlements, there is much less diversity of SDG representation (median: 4 SDGs per settlement, mean: 4, standard deviation: 3) compared to refugee settlements (median: 6), with 10 non-refugee settlements only having information on SDG 4 (Quality Education) (Figure 13). As with refugee settlements, SDG data are dispersed across non-refugee settlements, though select SDG features, such as those associated with SDG 15 (Life on Land), tend to spatially cluster (Appendix A Figure A2).

SDG data were created as early as December 2009 in non-refugee settlements; however, it was not until January 2016 when more than one hundred features were created in a given month (Figure 14a). Of these, 58% (1105) have remained V1 while 42% (814) have been edited at least once, which is more than twice the portion of edited SDG data in refugee settlements. While there is more sustained and consistent creation of nodes at refugee settlement nodes, there are similar patterns of concentrated periods of V1 data creation in refugee and non-refugee settlements. For example, the three-month period of January through March 2016 saw the collection of 46% of all SDG data in non-refugee settlements as well as the terminal versioning of 41% of all features that had been collected by that time (Figure 14b). For most SDGs, the majority of OSM data were created rapidly over a one- to two-month period (Figure 15). Only for SDGs 9, 15, and 16 were data collected gradually over many months and years, which is more typical of SDG data accumulation timelines in refugee settlements (Figure 10).

Versioning for SDG data in refugee and non-refugee settlements were comparable with a median of two versions for a given SDG (Figure 16a). SDG 9 (Industry, Innovation, and Infrastructure) stood out with a median V1–Vn duration of 1000 days (Figure 16b), more than double any other SDG’s median duration in non-refugee settlements and more than three times the median duration of any SDG in refugee settlements (Figure 11b). Indeed, all SDGs in non-refugee settlements had a longer median V1–Vn duration compared to their refugee settlement counterparts, which is expected because of the years earlier start of OSM data creation in non-refugee settlements. While SDG 5 data in refugee settlements benefits from high frequency and long duration, there are no SDG 5-related OSM features present in non-refugee settlements.

5. Discussion

Given the recent and historic increase in the global refugee population and the continued growth of the OSM data archive, this study offers timely and novel insights into the value of open geospatial data for SDG monitoring at refugee settlements. Using a case study of 28 refugee settlements in Uganda and 26 nearby non-refugee settlements, we find broad spatial and thematic representation of SDGs in refugee settlements, albeit with data that are generally out of date or were never updated. OSM data provided information on 11 of 17 SDGs, with information on six SDGs at more than half of refugee settlements and a particular abundance of OSM data on SDG 6 (Clean Water and Sanitation) and SDG 8 (Decent Work and Economic Growth). Refugee settlements not only had a larger count of OSM features than nearby non-refugee settlements, but a greater portion of these OSM data in refugee settlements was relevant for SDG monitoring. However, 64% of SDG-relevant OSM data at refugee settlements were collected in the single month of May 2018, 86% of data were never updated after their initial creation, and the creation of SDG-relevant OSM features typically did not begin until three years after refugee settlement establishment. Such concentrated periods of SDG data collection years after settlement establishment makes it difficult to monitor long-term SDG progress.

While the relative richness of SDG-relevant OSM data in refugee settlements is a positive sign for SDG assessments, survey data collected by the Ugandan government indicate that refugees lag behind the general Ugandan population on SDGs due to persistent poverty (SDG 1), widespread food insecurity (SDG 2), lack of clean water (SDG 6), expensive and harmful energy production (SDG 7), and constrained employment opportunities (SDG 8) [46]. Below, we identify four factors that contribute to the divergence between this study’s OSM-based SDG results and survey-based SDG progress within Ugandan refugee settlements. Considering these factors, we contextualize the specific opportunities and limitations of OSM data for SDG assessments in refugee settlements and identify ways forward to improve future OSM-driven SDG assessments in refugee settlements in Uganda and beyond.

5.1. Counting SDG-Relevant OSM Features Is Not the Same as Measuring SDGs

The enumeration of OSM features is not always a meaningful measurement of SDG progress or development in general [59]. The reliance on counting leads to an overrepresentation of physical features that can be located and quantified, but which may have little functional bearing on SDG progress. Moreover, relying on feature counting results in the appearance of greater SDG progress in larger refugee settlements (for example, see Figure 17), even though the larger count is likely a consequence of more OSM data creation effort having been directed to more populated settlements. Without a contextual understanding of the place-based relationships between OSM feature counts and SDG progress, each additional OSM feature is naively interpreted as being representative of incremental SDG progress. For example, the creation of new OSM features representing toilets does not necessarily mean that progress has been made on SDG 6 (Clean Water and Sanitation). The existence of a school building does not provide information about staffing or the quality of educational outcomes relevant for SDG 4 (Quality Education). Knowing a health clinic’s location does not aid in understanding the demand for medical services or health provisioning per person, which relate to SDG 3 (Good Health and Well-Being). Further, the existence of any physical feature recorded in OSM says nothing about access to or use of that feature. While OSM data provide information on the existence of resources, they are agnostic about the formal and informal power relationships and inequalities that determine people’s ability to access and use those resources, which are fundamental to assessing and monitoring SDG progress [60]. Likewise, the absence of mappable features does not necessarily equate to a lack of related SDG progress since some SDGs may not be well-represented by a geographic feature (e.g., SDG 10 on Reduced Inequalities).

5.2. OSM Data Provide SDG Status Snapshots Rather Than SDG Progress

OSM data collected in rapid, brief periods provide snapshots of SDG status but offer less insight into the progress of development over the study period. SDG progress is assessed by charting the creation of new data or revision of existing data. However, 81% of SDG-relevant OSM data in refugee settlements were never updated after initial creation, and those features that were updated were usually only updated once and most often within the same month as their creation. The creation of OSM data also lagged well-behind settlement establishment and refugee arrivals, and new data creation and terminal versioning concluded six months before the end of the study period. This means that the earliest available SDG data tend to also be the most recent data, resulting in a static, permanent depiction of SDG status rather than progress. The lack of versioning combined with the delayed initial creation and early completion of OSM data collection means that SDG progress cannot be reliably measured over the study period and any outcomes of development interventions by refugees, the government, or humanitarian actors are overlooked.

5.3. SDG Mapping Is Based on OSM Data Created for Humanitarian Mapping

In refugee settlements, OSM data have been created in support of humanitarian goals of meeting basic human needs (e.g., water, sanitation, and hygiene). The “development derivative” of these data—that is, the development-relevant information derived from humanitarian-oriented mapping efforts—are nonetheless valuable for mapping SDGs, especially those goals that are concerned with immediate human needs such as SDG 6 (Clean Water and Sanitation) and SDG 2 (Zero Hunger). On the other hand, SDGs that are less relevant for humanitarian concerns may have minimal or no representation by OSM data. For example, we lack OSM data on SDG 7 (Affordable and Clean Energy), SDG 10 (Reduced Inequalities), SDG 12 (Responsible Consumption and Production), SDG 13 (Climate Action), SDG 14 (Life Below Water), and SDG 17 (Partnerships for the Goals), and have only minimal representation of SDG 5 (Gender Equality) and SDG 16 (Peace, Justice, and Strong Institutions). Even though there may be on-the-ground evidence of these SDGs in refugee settlements, we lack the OSM data for their documentation. Further, concerted humanitarian-driven data creation in refugee settlements augments the appearance of SDG progress in refugee settlements relative to nearby non-refugee settlements that did not receive similar humanitarian attention (see Objective 3 Results above). This gives the impression that refugee settlements are outpacing non-refugee settlements with regard to SDG progress even though, in reality, refugees in Uganda lag far behind the general population in SDGs progress [46].

5.4. OSM Data Do Not Capture Aspatial and Relational Aspects of Sustainable Development

Using OSM data to monitor SDGs is an imprecise way to meaningfully measure sustainable development in refugee settlements. OSM documents physical features in the natural and built environment, but the SDGs are also informed by aspatial and non-material indicators of poverty, equality, and justice, as well as spatially continuous indicators associated with population density or air or water quality; for example, vector-based geometric primitives used by OSM cannot embody such complex spatial and aspatial relationships. Relatedly, OSM data do not capture the relational dimensions of sustainable development in refugee settlements such as the interactions and interdependencies between refugees and nearby communities. Refugee populations are often socioeconomically intertwined with nearby settlements as consumers and producers that sustain local economies, and as trade partners and a labor force [61,62,63,64], which are important non-material dimensions of economic development [65].

5.5. Recommendations for Future Work on SDG Monitoring in Refugee Settlements with OSM Data

This study provided valuable insights into the value and limits of gauging SDGs in refugee settlements using OSM data. With the SDG-OSM data model (Appendix A Table A1) developed in this study and fully open data from OSM and GRID3 (currently only available in Africa), this study’s approach could be transferred to other refugee-hosting countries. In an effort to make OSM more valuable for future efforts towards monitoring SDG progress in refugee settlements, we offer three recommendations. First, developing and systematically adopting OSM tags associated with specific SDGs and individual SDG indicators and targets would improve the clarity and fidelity of SDG-OSM relationships. In this study, we established an SDG-OSM data model based on available OSM tags within study refugee settlements (Appendix A Table A1). For simplicity, we selected a single tag for a given feature that was most representative of the feature’s function and had the clearest linkage to an SDG. We also only considered relationships to specific SDGs rather than the several targets and indicators for each SDG in this initial study. Going forward, developing a standardized, fully developed SDG-OSM data model or having one or more SDGs or associated targets or indicators expressly identified in an OSM tag (or tags) would remove the need to infer such relationships on a case-by-case basis and support consistent use and comparability regardless of geographic region or context. While there has been progress toward developing an Open Mapping for SDGs guide (sdgs.hotosm.org (accessed on 1 December 2020)), many SDGs still lack relationships to OSM data, other SDGs are incomplete, and relationships to targets and indicators are not evident.

Second, collecting data on refugee needs and priorities through direct (and ideally participatory) engagement with refugees would provide a more grounded OSM-driven characterization of sustainable development within refugee settlements, i.e., Reference [52]. Refugees are not merely passive recipients of aid, and SDG-relevant OSM features mapped in refugee settlements are not only a product of top-down initiatives from the host country or international community [66]. Given appropriate opportunities and the removal of constraints, refugees drive and sustain development through their own self-organized initiatives, including literacy and higher education programs [67], reproductive health programs [68], and entrepreneurship [69,70,71]. In the absence of refugee-led data collection for SDG monitoring through direct, participatory involvement of refugees (e.g., survey, interview, photo-voice methodologies), OSM can only offer limited insights into the self-identified needs and priorities of refugees with regard to their development. Recent development of the Participatory Mapping Toolkit is very promising in this regard [25]. While the Toolkit explicitly engages refugees to create OSM data, it remains unclear the extent to which refugees access and use OSM data in self-directed development-related decision making. Integrated data on OSM features and qualitative descriptors would thus be useful not only for articulating SDGs progress, but it would also have the potential to dispel perceptions of (1) refugee needs and priorities conforming to those identified by the international community for the average global population, and (2) refugee dependence (and drains) on international and national support systems.

Finally, using OSM data in conjunction with remotely sensed data at and surrounding refugee settlements would compensate for scarce OSM data for some SDGs (as shown in this study) and also improve the duration and frequency of SDGs monitoring [72,73]. This study showed that OSM data within Ugandan refugee settlements were broadly fixed in time, however public (non-commercial) satellite imagery from NASA’s (National Aeronautics and Space Administration) Landsat and ESA’s (European Space Agency) Sentinel constellations as well as private (commercial) imagery from Planet’s and Maxar’s growing fleets of sensors provide regularly updated (i.e., daily through sub-monthly) SDG-relevant data beginning as early as settlement establishment and continuously through settlement occupation [74,75,76]. Moreover, the consistency and stability of satellite data collection means that SDG-relevant data can be immediately compared over geographic space (i.e., between settlements) and over time. The use of satellite remote sensing data for monitoring SDG 6 [77], SDG 8 [78], and SDG 11 [79,80,81], in particular, offer a complementary landscape-based perspective to OSM’s feature-based perspective. While the application of satellite imagery for monitoring landscape changes at or surrounding refugee settlements is relatively recent, e.g., References [82,83,84], an SDG monitoring effort that fuses OSM and satellite remote sensing data would offer a scalable, near-real-time, persistent approach, well-suited for refugee settlement-level SDG monitoring.

6. Conclusions

This study provides the first systematic examination of the value and limitations of using OSM data for monitoring SDG progress in 28 Ugandan refugee settlements. The thematic breadth and broad spatiotemporal reach of OSM data supported mapping 11 different SDGs throughout Uganda’s refugee settlements and tracking the accumulation of SDG-relevant OSM data in refugee camps over an approximately four-year study period. Even though these data were not collected within an SDG mapping framework, the inherent flexibility of OSM data supported this rich, detailed investigation into settlement-level SDG progress. However, these positive outcomes are balanced by the punctuated rather than continuous mode of SDG data creation and versioning and the absence of data on six SDGs. These limitations highlight the need to evaluate potential sources of bias—consistent over- or under-representation of SDG progress—in settlement-level OSM data availability before undertaking any OSM-driven SDG assessment in refugee settlements, especially when comparing OSM-based results to survey or remote sensing-based results or comparing SDG progress across multiple settlements.

This study has several limitations that can be improved upon in future research. We created an SDG-OSM data model relating a single OSM tag to a single SDG based on available OSM data in study settlements. Similar to the Open Mapping for SDGs data model (sdgs.hotosm.org (accessed on 1 December 2020)), we did not consider potential many-to-many relationships between multiple OSM tags and multiple SDGs, nor did we identify specific SDG targets or indicators associated with one or more OSM tags. Had we considered a plurality of SDG-OSM relationships at the target or indicator level, we would have identified more SDG-relevant features and potentially more SDGs and could have developed a more nuanced understanding of the SDG importance of various features, at the obvious expense of increased complexity. We also lacked refugee settlement-level validation data on SDG progress with which to evaluate our findings and instead could only engage with national level reporting—this settlement-level data gap partially motivated the study in the first place. Finally, we only considered the nearest non-refugee settlements for comparison with refugee settlement SDG-relevant OSM data. Since these comparison settlements generally had far less OSM data than refugee settlements, examining more distant settlements with similar OSM data density could have offered a more informative comparison.

The widespread availability of OSM data make it a promising source of information on SDGs in refugee settlements beyond Uganda’s borders and in other regions with pervasive data scarcity, such as peri-urban informal settlements (so-called “slums”) and internally displaced person (IDP) encampments. In order to advance the utility of OSM data for SDG mapping, we identify the need for a standardized SDG-OSM data model suitable for mapping development-relevant places and spaces as defined by refugees themselves. Relational aspects of sustainable development in refugee settlements that are difficult to encapsulate in place-based OSM datasets, such as refugee–host economic relationships, or landscape-level characteristics, such as food security or transportation infrastructure, merit further attention. Future research would do well to examine the potentially unique advantages of field survey, remotely sensed, and OSM data for monitoring SDG progress in settlements. By understanding the relative strengths and complementarity of these three modes of SDGs data collection in a participatory framework, a spatially and temporally continuous approach to SDG monitoring that is simultaneously grounded and scalable could be developed. With the global refugee population growing every year, an integrated approach would be well-positioned to provide the localized, long-term data needed to support sustainable development in refugee settlements through 2030 and beyond.