RUS: A New Expert Service for Sentinel Users  ^†

Abstract

With large volumes of data acquired every month, the Copernicus satellites provide essential information for analysing and monitoring our environment. However, technical and knowledge barriers may affect user’s uptake of such a wealth of information. The RUS (Research and User Support for Sentinel Core Products) Service (funded by the EC and managed by ESA) began operations in October 2017 and aims to support overcoming such issues. A scalable cloud environment offers the possibility to remotely store and process data by bringing data and associated processing closer to the user. An integral part of the solution is the exploitation and adaptation of the platform, Free and Open-Source Software (FOSS). In addition, technical and scientific support (including training sessions) are provided to simplify exploitation of Copernicus data. The RUS Service is specially addressed to users from Copernicus countries who are willing to discover and use Copernicus core products and datasets. Other users willing to access the Service should first liaise with RUS to check their eligibility. The service is free. Commercial and operational activities cannot be carried out through the RUS Service.

Published: 23 March 2018

1. Introduction

In November 2016 the monthly volume of data acquired by the three operational Copernicus satellites (Sentinel-1a, Sentinel-1b and Sentinel-2a) accounted for 150 TB [1]. With the launch of Sentinel-2b, Sentinel-3a and Sentinel-5P this volume of data has at least tripled, implying that with a download speed of 15 Mps (average connection speed in Europe [2]), almost 8 years would be needed to download one month of all observations. In addition, a very performant computing power is needed to process such data (whose size, in the case of Sentinel-1 might be larger than 1 GB/product and 500 MB/product for Sentinel-2). Finally, besides “physical barriers” there might be “knowledge barriers” related to the complexity of the image information, understanding of the formats, the applicability of the data to specific applications or reluctance to absorb the data into pre-existing routines managed by the user.

RUS (Research and User Support for Sentinel Core Products) was launched with the purpose of helping to overcome these problems. The service is offered at no cost and addresses the needs (in terms of technical and scientific support, computing resources and disk space identified by ESA) of different types of users (basic users in need of downloading support; R&D users in need of prototyping support and proficient users, in need of processing support).

2. Methods

The following paragraph provides an overview of the methods and solutions put in place by the RUS consortium to mitigate the problems faced by Copernicus users.

2.1. ICT Solutions

To overcome physical issues (e.g., downloading, storing and processing), the service exploits Infrastructure as a Service (IaaS) and Platform as a Service (PaaS). IaaS includes network access, Virtual Machines (VM) with Computing Processing Units (CPU) and a scalable storage capacity. The PaaS includes data access (direct access to Copernicus Hub), communication tools (mail, chat, audio-conference and video-conference with the Helpdesk), processing and viewing tools, development tools, collaboration tools, as well as all necessary and relevant documentation and internet links. FOSS is pre-installed on demand on the VM, however users are free to install their Commercial Off-The-Shelf (COTS) software on the machine.

The infrastructure relies on several types of virtual environments:

• Collaboration environments hosting a platform to offer collaboration services such as video-conference and chat, the Front Desk, the Administration Desk and the Service Management Desk.
• User environments hosting the development and processing platform: each RUS user could have access to a dedicated cluster of user environments.

Thanks to this environment, RUS Users can access Sentinel data using the data platform, develop algorithms and process this data using their dedicated cluster and benefit from interactive support from RUS Operators through services offered by the collaboration platform.

Use of Copernicus datasets as the main source of information is a prerequisite to access the RUS Service, but non-Copernicus data (EO and other data) can also be freely used and imported by the users. The VMs provided by RUS work on a Linux environment where either FOS or COTS can be installed and also includes programming and scripting environments. Default Processing libraries account for: GDAL, Sentinel Toolboxes, Orfeo Toolbox and SNAPHU; pre-installed processing tools include QGIS and SNAP, whereas current software development utilities are: Oracle JDK 1.8, Apache Ignite, Eclipse, GCC, CMAKE, Maven, GIT, Python 2.7/3.5, and R 3.3. The ICT for the user is defined following an analysis of the received service request; such analysis defines the scaling of the work environment in terms of duration, disk space and size (number of Virtual Machines, number of cores per machine, RAM per core).

Considering resource constraints, the RUS Service can be offered to each user for a limited amount of time and including ICT/Expert/Data resources compatible with declared uptake objectives and current user demand. Three pre-defined work environments are typically proposed: 1–4 cores with disk space up to 1 TB for 3 months, 1–10 cores with disk space up to 10 TB for 6 months or up to 40 cores with disk space up to 50 TB for 6 months.

More information about the RUS Service and access to the VM can be found at: https://rus-copernicus.eu/portal/.

2.2. Building Knowledge

To complement the ICT offer, training and outreach activities aiming to create a critical mass of Copernicus data users and focusing on a large portfolio of applications, are side supporting activities surrounding the service pyramidal layers. Use of the RUS Virtual Machines with pre-installed FOSS facilitates handling of such events, where participants can use their own laptops to manage the processing. The use of the same configuration for each VM in fact discards any pre-existing difference between the used laptops (and operating systems), facilitating the smooth running of the event. Face to face events are organized to meet the requirements of small groups of users which receive specific training on EO theory and then are guided step-by-step by the trainers, in the application of the learned theory in practical case studies. The assigned VM remains accessible to the user for several months after the training, so as to allow repeating or completing the exercises (or performing other processing activities).

Large Webinars organized every month aim to attract new potential users by providing in a condensed format, the instructions to perform some basic processing steps to exploit Sentinel data for a specific application. They are closed by Q&A sessions, offering the participants the possibility of interacting with the trainer. The Webinars are recorded and made publically available for re-play on a dedicated YouTube channel. Users interested in repeating the exercise can either use FOSS installed on their computer or ask RUS for access to the pre-configured VM with all the material needed to perform the exercise.

The theory lectures given during the face to face events are recorded and assembled with questions and multiple-choice answers and are made available on an E-learning portal. Scores are assigned for each completed course and badges are given to the users.

More information and access to RUS Training Resources can be found at: https://rus-training.eu/

3. Results and Discussion

In this paragraph we provide a few examples of processing results focusing on different applications, obtained by exploiting the service to prepare training sessions. The data and software needed to re-play the exercises are freely available within the RUS environment, together with the step-by-step instructions to generate most of the presented results.

3.1. Ship Detection

Ship detection with Sentinel-1 enables detection of vessels not carrying an Automatic Identification System (AIS) or other tracking systems on board, such as smaller fishing ships or ships that might be in the surveyed area illegally (illegal fishing, piracy etc.). As SAR is not reliant on solar illumination and is rather independent of weather conditions, frequent monitoring is possible. The exercise exploits ESA’s Open Source Sentinel-1 Toolbox to process Sentinel-1 data, detecting targets larger than 30 m in the Gulf of Trieste. Final output can be visualized in Google Earth (Figure 1) or be exported as a point layer to an Open source GIS (QGIS). RUS VM are used for running the exercise. More information about the use of Sentinel-1 data for maritime surveillance can be found in [3].

3.2. Burned Area Mapping

Two Sentinel-2 products acquired before and after a series of wildfires which affected central Portugal in June 2017 are used to map the location and intensity of damage (burn severity). The exercise exploits ESA’s Open Source Sentinel-2 Toolbox to process Sentinel-2 data, comparing pre- and post-event imagery and calculating the Relativized Burn Ratio (RBR) [4]. Processed results are then exported to an Open source GIS (QGIS)—Figure 2, where post-processing (classification of severity level, following USGS suggested classification) is performed. RUS VM are used for running the exercise.

3.3. Deformation Monitoring

Two Sentinel-1 images acquired before and after the Iran earthquake of 12 November 2017 are used to create the deformation map associated to the event. In this case ESA’s Open Source Sentinel-1 Toolbox is used to create an interferogram from a couple of ascending acquisitions and to derive line of sight subsidence/uplift associated with the event (Figure 3). RUS VM was used for processing.

3.4. Discussion

In the first two cases presented, lack of in situ observations simultaneous to the acquisition do not allow thorough validation of the results, therefore they should only be considered as examples of well-established methodologies. In the case of the deformations associated to the November earthquake, the distribution of fringes and estimated line of sight motion are well in agreement with studies carried out by other authors with the same datasets, as well as with different data [5].

Tutorials to reproduce the results described above for exploiting the RUS service are being made freely available on the dedicated RUS YouTube channel https://www.youtube.com/channel/UCB01WjameYMvL7-XfI8vRIA. Furthermore upcoming training events are announced through social media, such as Twitter (@RUS_Copernicus) and Facebook (https://www.facebook.com/RUS-Copernicus-1940884026129145).

4. Conclusions

The RUS Service is a new, free service carried out by an international team, led by C-S France and involving Serco SPA, Noveltis, Along-Track and C-S Romania. The main aim of RUS is to promote uptake of Copernicus satellite data. This is achieved by facilitating user access and exploitation of the data, through the use of VMs with associated processing power, and by carrying out training and education activities.