A Serverless-Based, On-the-Fly Computing Framework for Remote Sensing Image Collection

Abstract

The rapid growth of remote sensing data calls for the construction of new computational models for algorithmic exploration, which requires on-demand execution, instant response, and multitenancy. We call this model on-the-fly computing, which could reduce the complexity of cloud programming for remote sensing data analysis and benefit from efficient multiplexing. As an advancement of cloud computing, serverless computing makes it possible to realize the on-the-fly computational model. In the study, the concise definition of an on-the-fly computing model for remote sensing data analysis and the corresponding software architecture based on the serverless computing commodities are presented. The proof-of-concept experiments have suggested that the on-the-fly computing model for remote sensing data analysis can be efficiently implemented as a serverless software. The response time is mainly related to the tile reading operation and data structure conversion. In the case of high concurrency, the system can scale to hundreds of instances in seconds.

1. Introduction

With the advance in earth observation and surveying technology, remote sensing images are increasingly accumulated and piled to be processed, heading the community of geographical information science into an era of big data. Massive remote sensing images, multisource archives of petabytes, pose great challenges for the traditional geospatial information analysis infrastructure. Cloud computing is one of the most promising technologies to tackle these challenges, and the geographical information science community has developed various cloud computing platforms for massive remote sensing images analysis [1,2].

Among the existing massive remote sensing images analysis frameworks, the most influential one is Google Earth Engine (GEE) [3], which provides two types of computing services, namely on-the-fly computing and batch computing. Based on the traditional big data analysis techniques, batch computing processes all the input data as a whole [4]. As remote sensing data grows larger and larger, the execution time of batch computing is becoming longer and longer. However, exploratory analysis in scientific research usually requires an environment enabling instant read–eval–print loop (REPL), where a program is executed piecewise for a rapid result evaluation. To solve this problem, the on-the-fly computing technologies, featuring on-demand execution, instant response, and multitenancy, are developed to improve the data science productivity. Although GEE provides an excellent instance of the on-the-fly cloud computing paradigm and has been widely used by the remote sensing community, its theoretical basis, design principles, and implementation details are not publicly available.

General big data analysis frameworks, such as Hadoop [5], Spark [4], and Dask [6], are based on individual servers with tightly integrated resources, which is called server-centric computing. As advancement in the computer architecture community enables the datacenter disaggregation [7], serverless computing comes to light [8]. Serverless computing brings about cloud functions with greater elasticity and more lightweight virtualization while changing the pricing of cloud computing from paying resources allocated to paying in proportion to resources used. Unfortunately, to the best of our knowledge, none of the existing on-the-fly remote sensing image analysis frameworks, like Geonotebook [9], have yet adopted serverless computing technologies and could not switch to batch processing seamlessly at the same time.

In summary, this paper makes the following contributions:

(1)

Proposing a definition for the on-the-fly cloud computing paradigm for remote sensing image collections, including some empirical or descriptive characteristics and a formal definition.

(2)

Designing an entirely serverless architecture based on the serverless commodities of a public cloud, which consists of a data model, a programming model, and a series of key implementing technologies for remote sensing image collection analysis.

(3)

Providing some concrete, proof-of-concept experiments suggesting that on-the-fly cloud computing for remote sensing images can effectively run on the serverless cloud platform.

The remainder of this paper is organized as follows: Section 2 is an overview of the definition for the on-the-fly computing paradigm and introduces its serverless software architecture. More details about the implementation are presented in Section 3, Section 4 and Section 5, and Section 6 shows some concrete proof-of-concept experiments of on-the-fly computing for remote sensing images. Section 7 discusses the results and concludes the paper.

2. On-the-Fly Cloud Computing

2.1. Cloud Computing vs. HPC

Currently, cloud computing has become the main paradigm of server programming, which can ship code to the big data. The key technologies include virtualization, distributed storage, and distributed computation. A large number of frameworks have been developed by the industry and scholars, which can be classified into the server-centric pattern or serverless pattern. There are some serverless computing commodities in the public cloud, such as AWS Lambda.

There are four requirements for any computing system, including ease of use, high performance, portability, and flexibility. The cloud computing system’s first object is the ease of use while that of high-performance computing (HPC) is performance. Therefore, HPC provides programming abstractions with low-level details about computer architecture, such as MPI, and cloud computing systems have more automatic optimization mechanisms.

From the perspective of workload, the big data processing frameworks can be classified into batch processing and streaming processing. However, on-the-fly computing has a significant difference from the other two paradigms. The data source for on-the-fly computing is the same as batch processing, but it requires a quick response. The streaming processing can respond instantly, but its data source is real time. Therefore, existing, general cloud computing frameworks cannot be directly applied to the algorithm exploratory analysis. A new paradigm of geocomputation is needed.

2.2. Characteristics

The target computing model is called on-the-fly cloud geocomputation, which is implemented based on general purpose cloud computing technologies, oriented to exploratory analysis, and dedicated to remote sensing processing. We summarize the characteristics of on-the-fly cloud geocomputation from the perspective of human–computer interaction as shown in Figure 1.

(1)

Shipping code to the remote sensing images persisted in the cloud storage instead of downloading the data locally for analysis.

(2)

Seamlessly switching to batch processing without code modification, which requires the data abstraction and operators to be the same.

(3)

Implicitly triggering the execution implied in specific operators, such as visualization and data export.

(4)

Dynamically determining the spatial scope of remote sensing images to be processed based on the tiles visualized on the map.

(5)

Responding as rapidly as possible when the user needs to evaluate without queueing of workloads.

(6)

Executing user-defined codes based on the overviews of remote sensing images without explicitly provisioning and managing data allocation.

(7)

Paying in proportion to remote sensing data used instead of paying for the computing resources allocated.

2.3. Formal Definition

The empirical and descriptive characteristics of on-the-fly cloud geocomputation are not sufficient to determine the structure of the target model. A directed acyclic graph (DAG)-based model of on-the-fly cloud geocomputation is presented in Definition 1. A DAG is a kind of intermediate representation for user-defined codes, and it is common in relational databases, where it is used to represent the query plans. The nodes of a DAG are function invocations with some edges for representing the inputs and outputs. Operators and user-defined codes are equivalent logically and can both be transformed into DAGs in the target framework.

Definition 1.

In order to simplify cloud programming, the target framework of the on-the-fly cloud computing model for remote sensing images should provide rich datatypes, analysis-ready data, and dedicated operators for remote sensing image analysis, which would significantly reduce the amount of user-defined codes. Its programming model could only acquire and generate the datatypes and operators accessible, according to user authentication in the frontend, and process data visible to users on maps, usually in the form of tiles, finally achieving instance response, on-demand execution, and mutitenancy.

The on-the-fly computing model for the remote sensing data analysis $M$ can be defined as a five-tuple:

$$M\colon\colon \left(E,S,U,V,A\right)$$

Here notation $\colon\colon$ means “defined as”, and $E$ refers to the main elements to be processed, including operators, datatypes, and DAGs, which can be defined as a set:

$$E\colon\colon D{\displaystyle \cup }OP{\displaystyle \cup }G$$

$D$ refers to the set of datatypes, also known as data models, commonly used in remote sensing image processing, such as $Image$ and $ImageCollection$. Each datatype $D_i$ is a set of elements $d_i$, which means that $d_i$ is a specific dataset of type $D_i$.

$$\begin{gathered} D\colon\colon \left\{D_i\right|i\in N\} \\ {D}_i\colon\colon \left\{d_i^k\right|k\in N\},i\in N \end{gathered}$$

There are some specific relations $R$ between datatypes in $D$, which can be defined as a three-tuple. A predicate $p$ is virtually a functional mapping from one datatype $D_i$ to another datatype or operator. Superscript $\ast$ means receiving the power set.

$$r\colon\colon \left(\alpha ,p, \beta \right)\iff p\left(\alpha \right)\to \beta ;\alpha \in D{\displaystyle \cup }OP,p\in P,\beta \in D$$

Here notation $;$ means the end of an expression. There are only two predicates. The predicate $cp$ means that $\alpha$ is a component of $\beta$, and more complex datatypes can be established through it. The predicate $ih$ means that $\beta$ is more specific and customed on the basis of $\alpha$. We can build a classification based on predicate $ih$. It should be pointed out that all the datatypes and operators can be organized as a network through these two predicates.

$$\begin{gathered} cp\left(\alpha \right)\to \beta \Rightarrow \alpha \in \beta ;\beta \in D,\alpha \in D{\displaystyle \cup }OP \\ ih\left(\alpha \right)\to \beta \Rightarrow \beta \in {\alpha }^{*};\alpha ,\beta \in D \end{gathered}$$

$OP$ refers to a set of operators dedicated to remote sensing image analysis, which is also known as the programming model. Each operator $op$ is bounded with a specific datatype $D_i$ through the predicate $cp$. Every operator is actually a function mapping from certain datatypes with or without a base operator to the output datatypes.

$$\begin{gathered} OP\colon\colon \left\{o{p}_i\right|i\in N\} \\ op\colon\colon D^{*}\times OP\to D^{*} \end{gathered}$$

$G$ refers to a DAG, which represents the computational process of remote sensing images. The DAG is constructed by a series of computational nodes $n_j$, which represent functional calls with the output results $\alpha$, operator name, and input arguments. Similar to an operator, $G$ is virtually a function mapping from the input remote sensing data to the results. In the implementation of the framework, $G$ can be modeled as a series of nested objects. Each of them records the input arguments, operator name, and returned datatype.

$$\begin{gathered} G\colon\colon \left\{n_j\right|j\in N\} \\ n\colon\colon \left(\alpha ,op,\beta \right),\alpha \in D^{*},\beta \in G^{*}|D^{*} \end{gathered}$$

$S$ refers to a set of states of the main elements, including datatypes, operators, and the DAG. Any datatype, operator, or DAG can be located only in the client or on the server, and it can be a static string, a callable proxy, a piece of code, or an executable cloud function. Notations $st$, $cb$, $cd$, and $ex$ represent that the element is static, callable, code style, or executable, respectively. Notations $ct$ and $sv$ mean that the element is located in the client or server, respectively.

$$S\colon\colon \left\{st,cb,cd,ex\right\}\times \left\{ct,sv\right\}$$

The notation $U$ refers to end users. Each user has permitted access to certain operators. $V$ refers to the viewpoint on a map, which can be defined as a set of tile numbers. The viewpoint determines the spatial scope of input remote sensing data to be processed.

$$\begin{gathered} U\colon\colon \left\{u_i|i\in N\right\},u_i\in O{P}^{*} \\ V\colon\colon \left\{\left(x,y,z\right)\right|x,y,z\in N\} \end{gathered}$$

Notation $A$ refers to a set of actions to change the state of target elements to complete the whole computational process.

$$\begin{gathered} A\colon\colon \left\{get,init,gnrt,sbmt,schdl\right\} \\ a\colon\colon U\times S\to U\times S,a\in A \end{gathered}$$

The action $get$ can change the location of some elements that can be accessed by a certain user $u_i$. Notation$:$ means value of the state $S$.

$$get\colon\colon u_i\times S:\left(st,sv\right)\to u_i\times S:\left(st,ct\right),u_i\in U$$

The action $init$ represents an action for the initialization of datatypes and operators, which translates the state of them from static to callable. A datatype or an operator that is callable means that it can be programmed but will not be actually executed.

$$init\colon\colon u_i\times S:\left(st,ct\right)\to u_i\times S:\left(cb,ct\right),u_i\in U$$

Notation $gnrt$ represents an action for DAG generation, which translates the user-defined script to a DAG object. The state of the DAG changes from code style to callable.

$$gnrt\colon\colon G\times S:\left(cd,ct\right)\to G\times S:\left(cb,ct\right)$$

In contrast to action $get$, $sbmt$ represents the action of the DAG submission, which can be modeled as translating the DAG to a static string and changing the location of the DAG from client to server.

$$sbmt\colon\colon G\times S:\left(cb,ct\right)\to G\times S:\left(st,sv\right)$$

The action $schdl$ represents the action of DAG scheduling, which changes the state of the DAG from static to executable and obtains the result tiles determined by viewpoint. The execution or scheduling of DAG depends on a run-time environment, which can be modeled by a process calculation [10]. Serverless has no formal foundation yet, and to simplify the definition, we do not model the execution details of the DAG in the backend.

$$schdl\colon\colon G\times S:\left(st,sv\right)\times V\to \left\{til{e}_i\right|i\in V\}$$

It should be noted that the essence of the element state change is a process of translation rather than a process of encapsulation and invocation.

2.4. Serverless Architecture

In this study, a pure serverless software architecture means that all the components are built on serverless commodities from the public cloud providers, mainly including the function computing (FC) [11], serverless workflow [12], Tablestore [13], message service (MNS) [14], relation database system (RDS) [15], and object storage service (OSS) [16] of Alibaba Cloud. This architecture is shown in Figure 2, which introduces the high-level components in the target system and traces the execution flow of the UDF creation and pipeline execution. Due to the adoption of serverless technologies in software design, the cost of the system can be paid after the construction is completed. This will enable flexibility in the pricing of system services.

① User-defined function (UDF) creation. Some basic operators are defined and submitted to a cloud function through the UDF client. This cloud function registers the UDF in the FC engine and a symbol database. As operators are dynamically generated, users can access the created UDFs through the pipeline client.

② Control flow of execution. User-defined code based on the pipeline client are expressions that can be translated to DAGs. These DAGs are submitted to the receiver based on a cloud function and then persisted in the DAG run-time storage. Another cloud function is triggered by the run-time storage update events to translate the execution-ready nodes to a parallel workflow.

③ Data flow of execution. Remote sensing images in the serverless storage OSS can be asynchronously accessed and processed by executors. Tiles of the results are returned to the receiver and visualized on the map in the frontend.

3. Data Model

3.1. Tiling

Because on-the-fly geocomputation emphasizes analyzing during visualization, the target system needs to constantly load only part of the target remote sensing images to memory, which means that it is necessary to divide the images into tiles at different levels and organize them as pyramids.

For the remote sensing images, Cloud Optimized GeoTIFF (COG) [17] is the most popular file format to build a pyramid for on-the-fly cloud geocomputation. Because the I/O time is supposed to be much less than the time of connecting to OSS, all the tiles of different levels that belong to the same band are organized together in a single GeoTIFF file. The tiles of the COG are usually organized in a sequence of a row major.

There are two kinds of tiles in the target system: one is for visualization and the other is introduced by the COG format for cloud storage. Notation $t_v$ and $t_s$ represent a tile for visualization or storage, respectively. The tile number $\left(x_v,y_v,z_v\right)$ is usually determined according to Web Mercator projection.

$$\begin{gathered} t_v\colon\colon \left(x_v,y_v,z_v,valu{e}_v\right) \\ {t}_s\colon\colon \left(x_s,y_s,z_s,valu{e}_s\right) \end{gathered}$$

In this paper, we highly recommend that these two tiling strategies are consistent and aligned to some extent. Images from different satellites may have different spatial projections and need conversions from $\left(x_s,y_s,l_s\right)$ to $\left(x_v,y_v,l_v\right)$; $\left(x_s,y_s,l_s\right)$ is usually tiling locally along the rows and the columns of image and not aligned to $\left(x_v,y_v,l_v\right)$.

3.2. Logical Region

Tile is the basic unit for storage and visualization. The region is a logical strategy of tiles grouping, which is aimed at the dynamic requirements of data-parallel execution. It can be modeled as a set of tile numbers, representing a spatially continuous coverage, and is the minimal unit for the algorithm design, task allocation, and geodata access.

The calculation from a region number $\left(x_r,y_r,z_r\right)$ to visualization tile numbers $\left\{\left(x_v,y_v,z_v\right)\right\}$ can be modeled as an affine function $f_t$. $a_x$ and $a_y$ represent the width and height of the region, respectively. $b_{rx}$ is an offset in region $r$ along $x$ while $b_{ry}$ is an offset in region $r$ along $y$. $z_v$ and $z_r$ are levels in the pyramid, and they usually have the same value. Obviously, the conversion $f_t^{-1}$ from $\left(x_v,y_v,z_v\right)$ to $\left(x_r,y_r,z_r\right)$ is a kind of integer modular operation.

$$\left(\begin{array}{c}{x}_v\\ y_v\\ z_v\end{array}\right)\stackrel{f_t}{\leftrightarrow }\left(\begin{array}{c}{x}_r\\ y_r\\ z_r\end{array}\right)\times \left(a_x,a_y,1\right)\mp \left(\begin{array}{c}{b}_{rx}\\ b_{ry}\\ 0\end{array}\right),a_x,a_y,b_{rx},b_{ry}\in N$$

The region is composed of two independent types of tiles, which can be transformed to each other by $f_c$. Due to the transformation of the projection, a tile in the COG may belong to two different regions. In order to prevent the tiles belonging to different regions from being processed multiple times, a tile-masking strategy was designed. The mask is a set of tiles, which can indicate whether the target tiles have been processed. In computation, the region is reformed as a unified larger tile that can ensure the correctness of focal operators by the overlapped zones, such as sliding windows. The logical region $LR$ can be defined formally as follows:

$$LR\colon\colon \left(T_v,T_s,f_t,f_c,M\right)$$

$T_v$ and $T_s$ are the set of $t_v$ and $t_s$, respectively. $M$ is also a cached set of $t_s$ to indicate the tiles that have been processed. The logical region is as shown in Figure 3. In the OSS storage, the bytes to be read are determined by some information, including bucket, image collection, image, band, region number, tile order, and overlap. The $region$ and $overlap$ define the minimal basic unit for distributed geocomputation. The $overlap$ is a dynamic value determined by the window operators. The spatial $order$, similar to the Hilbert curve or Z curve, determines the offset and length of bytes where the system begins to read.

3.3. Datatypes

As a kind of raster data, the remote sensing image is the main type of geodata and can be analyzed by a map algebra or an array algebra system. In addition, vector data is another type of geodata, often used to manage the image processing results. The logical region is a general purpose geodata abstraction for implicit parallel computing, which is not suitable for end-user programming and should be invisible to users. We propose a composite datatype of image collection based on the SpatioTemporal Asset Catalog (STAC) [18] and GeoJSON. The definition is as follows:

Image collection is the top-level data abstraction dedicated to remote sensing image processing and constructed based on predicates $P$ and some basic datatypes. All datatypes and their relations are shown in Figure 4.

4. Programming Model

4.1. Workflow

The DAG is virtually a workflow for modeling the computational process, where the nodes and edges represent operators and image collections. DAG generation, presented in Section 4.4, is usually separated from the end-user programming interface and invisible to users, which makes programmers avoid the burden of constructing a global DAG data structure for workflow. This paper proposes a design or definition for the interface of remote sensing processing workflow. The user programming interface consists of composite datatypes and workflow skeletons, which can be expressed as a two-tuple, $\left(composite types,skeletons\right)$.

(1)

$composite type$. The composite datatypes are the user-visible components of data abstraction, defined in Section 3.3, except $region$ and $tile$. In the construction of workflow, $ImageCollection$ is the most common datatype.

(2)

$skeleton$. Workflow skeletons are high-level operators representing the basic workflow semantics. There are six operators related to workflow construction, including $create$, $filt$, $integrate$, $transform$, $aggregrate$, and $show$. These skeletons are functions mapping from one image collection to another.

$$Skeleton\colon\colon D_{11}\times OP\to D_{11}$$

$Model$, $Condition$, $Relation$, $Base operator$, $Aggregrator$, $Reducer$, and $Scheme$ are kinds of user-defined functions, defined in Section 4.2. The definition is shown in Figure 5.

Every $skeleton$ has two parts, the lefthand expression 〈$lhs$〉 and the righthand expression 〈$rhs$〉. All the input datatypes of $skeleton$ are $ImageCollection$. The $create$ operator is used to load an image collection with a file path of cloud storage or a data model $Model$ describing the content of the certain image collection. The $Filt$ operator is used to construct spatiotemporal or regular conditions for filtering the input image collection. Because each image collection has a corresponding data model $Model$, the $Integrate$ operator provides a way to integrate different image collections with a $Relation$ between different data models. The $skeleton$ $Transform$ maps a base intraimage operator to every item of the input image collection while the $Aggregate$ operator maps a base interimage operator according to the $Aggregater$, which also groups the items of the image collection by the selected dimensions in $Model$, such as time or space. The $Show$ operator triggers the execution of the workflow and obtains the tiles of the input image collection visible to users.

4.2. User-Defined Function

The $skeletons$ are collection-level operators and lack the flexibility of image-level or pixel-level operations. According to the definition of $workflow$, there are six operators at the image level and pixel level, which can be defined and published by the user, including $Model$, $Condition$, $Relation$, $Base operator$, $Aggregrater$, and $Scheme$.

Similar to the constructor of programming language, $Model$ is a configuration for certain satellite images, including band number, coverage, resolution, spatial projection, and other attributes in a key-value form. The $Condition$ is a logical expression, especially the spatial-temporal topological relationship, for selecting the target images. The $Realtion$ is a rename operator, which specifies a unified model for the input image collections.

$Base operator$ refers to the image-level algorithm defined and implemented on the band-level and pixel-level interface. The band-level operators, band math, or map algebra can be regarded as some window operators on linear algebra. The $reducer$ defines some algorithms to integrate all the images in a collection into a single image along a certain axis, such as $space$, $time$, $bands$, or other metadata.

Besides all the above operators, there are also some operators related to publishing UDFs, such as $register$, which provide an interface for registering the basic information of UDFs, including results datatype, function name, input arguments, and description, etc. More details about operator publication are shown in Section 4.3.

The syntax description is based on the augmented Backus–Naur form (ABNF), shown in Figure 6.

For the reason of embedding into other programming languages as an internal domain-specific language (DSL) [19], the production rules adopt some symbols different from the regular ABNF [20]. All the symbols of the production rules in Figure 4 and Figure 5 are defined in

Table 1. Description of the symbols.

Symbols	Description	Example
<>	Denotes an operator or variable in programming;	<img-clct>
[]	Indicates creating a data structure of numeric array;	[[…]…[…]]
{}	Body of the UDF or a data structure of dictionary;	{<metadata>*}
()	Indicates the inputs of the UDF;	(<img-lhs>)
|	Choice operator for two candidate expressions;	EQ | NE
*	Zero or more occurrences of the preceding element;	<img-op>*
+	One or more occurrences of the preceding element;	<img-op>+
.	Denotes the attribute or method of an object;	<id>.apply(…)
;	Indicates end of a BNF statement;	Image <id>;
//	Annotation of the production rules;	//annotation
::=	Means being defined as the right-hand expressions;	<l-op> ::= or;

.

4.3. Operator Publication

Similar to the symbol table of programming language, operators can be regarded as a kind of computational symbol with input and output information in the remote sensing processing workflows. An operator brings two kinds of information, one for user programming and the other for explicitly cloud functions calling, which can be called high-level attribute and low-level attribute of operators.

The high-level attribute is some information exposed to users, which would be sent to the pipeline client through a dynamic operator generation. It contains four kinds of information expressed as datatype, operator name, input arguments, return types, and description. The low-level attribute is a pointer to a certain cloud function, which contains two kinds of information, including operator name and function call location.

When users create operators, they need to publish in the backend before these operators can be used to create workflows. The process of operator publication is shown in Figure 7, presented in Python style. When UDFs have been published by users, they can access the published UDFs through agent datatypes through the pipeline client, and they can be used in the construction of workflows.

4.4. DAG Generation

The pipeline and UDF clients have the ability of dynamically generating operators in the frontend based on the metaprogramming technology and the high-level attributes of cloud functions. All the operators in the frontend belong to specific proxy objects, and they actually refer to one same function just recording the invocation between cloud functions, which make up a sense of code execution. This relation can be modeled as a DAG, which is a kind of intermediate representation between the remote sensing image processing pipeline and the underlying cloud functions, and generated through a series of callable proxy objects in the frontend, similar to the way of the GEE API Client.

4.5. Trigger of Execution

Although the DAGs represent the pipeline execution, they are generated in the frontend. When users invoke specific functions, the framework is triggered to submit DAGs from client to cloud. In order to receive results as soon as possible, the DAGs should have an initial viewpoint or would receive a default one if not. The framework can compute results on the tiles around the viewpoint. The trigger of execution can be defined as a tuple of $\left(function, viewpoint\right)$. The $function$ can submit the DAGs through a request and obtain a map identifier to fetch tiles. The $viewpoint$ defines the initial scope of input tiles. Therefore, the users can receive the result very quickly, complying with the design principle of on-the-fly geocomputation.

The most important aspect of DAG execution is to guarantee the correctness of translating the focal operators from band level to region level. For the reason that logical region is invisible to end users, the translator of the run-time environment should be able to determine the shape of logical region automatically, i.e., automatic data partition.

5. DAG Execution

5.1. Data Partition

Different from the DAGs generated automatically by the machine learning engine, the DAGs manually constructed by users are usually not too big to be partitioned for task-parallel execution. This paper only focuses on the algorithms of data-parallel execution. We need an algorithm for automatic data partition.

The focal operator has a characteristic of a structural locality, which means that the result at the location of $\left(i,j\right)$ is not only determined by the value of $\left(i,j\right)$ but also its neighbors. The structural locality of the remote sensing processing can be defined as a window so that the computational model of the data partition for a remote sensing image can be expressed as a function with a window as the input and a kind of logical region as the output.

$$region\leftarrow f_{image}\left(window\right)$$

Since all the remote sensing images are organized as a COG and persisted in the OSS, the data partition algorithm is ideally determined by the characteristics of OSS, the shape of the window, and the physical layout of the COG. As the COG is a highly customized format in the OSS for streaming, progressive rendering, and supporting on-the-fly random reading, the impacts of the OSS and the physical layout of the COG can be neglected in this paper.

A single remote sensing image can be regarded as a three-dimensional array, so the window defines the scope and neighboring pixels at a specified location of $\left(i,j,k\right)$. Different from the common window definition, such as 7 × 7 × 3, this paper adopts a more flexible window form, i.e., a set of neighboring pixels similar to ArrayUDF [21]. Expression $\left\{r,c,b\right\}$ represents the three dimensions of an image. The shape of the overlapping zone $\left\{O_k\right\}$ guarantees the correctness of operators across logic regions and can be derived from the window parameters, $\left[L_i,R_i\right]$.

$$\begin{gathered} W\left(\left[L_i,R_i\right]\right)\leftarrow f\left(\left\{P_{{\delta }_r,{\delta }_c,{\delta }_b}|{\delta }_i\in \left[L_i,R_i\right]\right\}\right),\forall i\in \left\{r,c,b\right\} \\ \left\{O_k\right\}\leftarrow Sum\left(L_k,R_k\right),\forall k\in \left\{r,c,b\right\} \end{gathered}$$

5.2. Execution

To implement computing while visualizing, we propose an architecture based on the producer–consumer pattern. The producer receives tile numbers indicating data to be processed, applies the DAG to be executed on tiles determined by regions, and puts the results in a workspace to be consumed. The data-parallel DAG execution algorithm is shown in Algorithm 1.

Algorithm 1 Data-Parallel DAG Execution

5.3. Cache

Considering that some tiles may be requested for more than one time in the DAG execution, the framework must maintain the execution states, including DAGs and tiles, in caches. The consumer firstly obtains tiles in the target region from the cache in the frontend and generates tiles dynamically from the upper or lower tiles maintained in the cache if they have been requested and then sends a request to the backend to receive the target tiles generated by the producers from the workspace.

Except for the cache in the frontend, there is also a cache in the backend, which plays an import role in the DAG execution. If the whole region requested by the consumers has been processed, the cache will put the tiles directly in the workspace. As some tiles are shared by different regions and the target region is partially processed, the cache will generate a mask to declare which tiles have been processed to reduce the computational cost.

6. Case Study

6.1. Data and Result

This study conceptually validates the feasibility of serverless-based, on-the-fly computation framework with a simple NDVI use case on a remote sensing image. The NDVI code is shown in Figure 8. All the remote sensing images are from Landsat8 and organized in the COG file format. Every overview of different levels in the image pyramid is tiled into tiles of 256 × 256 pixels and encoded into an individual TIFF file. This study provides a Python client integrated as an algorithm library in Jupyter. The NDVI code, the input image, and the result tiles are shown in Figure 8. The NDVI function is performed on four tiles, numbered as $\left(0,7\right)$, $\left(0,8\right)$, $\left(1,7\right)$, and $\left(1,8\right)$.

Before the system begins to process the target tiles, a DAG will be generated and submitted to the data center. Part of the DAG in Figure 9 is corresponding to the NDVI code from line 1 to line 3. It should be pointed out that the time point is modeled as a range from a start time to the end time. A DAG is actually a nested object returned by the final operator. The DAG generation is a process of creating this nested object.

6.2. Response Time

Once the algorithm is determined, it will be applied to the whole remote sensing collection. The response time is a critical characteristic of on-the-fly cloud computing for remote sensing data analysis during the phase of DAG construction or algorithm exploration. When users request to run the user-defined code, the frontend firstly submits a DAG and receives a task identifier, and then the backend executes it on the specified tiles determined by $VP$.

The response time is related to the efficiency of the scheduler, the complexity of the related algorithms, the latency of the communication, the characteristics of the serverless platform, and the concurrency. In the study, the response time is tested through a series of continuous requests shown in Figure 10.

The initialization phase of a serverless platform is called a cold start. The cold start of the NDVI case is around 1200 ms, which is about three times longer than that consumed by a base cloud function, which only responds to the request without doing anything. Once the serverless platform has been started, the response time will reduce to around 700 ms in the elastic mode whereas in the performance mode, the computation time will reduce to less than 400 ms. It is shown that the NDVI case benefits a lot from better hardware, but anything larger than a 4 GB memory could not further reduce the response time.

The response time of the NDVI case is mainly determined by reading tiles from the OSS, putting them together, and transforming them into an image. The time of reading a COG header and four tiles from the OSS is about 200ms, and the NDVI case needs to repeat this process. Then, the system will put four tiles together and transform them into an image, and it will take about another 200 ms. The scheduler only takes less than 100 ms, for the total response time is less than 700 ms. To reduce the response time, the system adopts a strategy similar to lazy evaluation, in which the operation of reading data from the OSS is only triggered by some specific operators that must manipulate the data. In the NDVI case, the operation of reading tiles from the OSS is deferred to the eleventh round of task scheduling.

6.3. Concurrency

An important characteristic of serverless computing and the requirement for on-the-fly cloud geocomputation is concurrency. Once the computation request is sent by a map client automatically, the serverless framework needs to respond to a large number of computation requests in a short period. The tested concurrency performance of the serverless-based remote sensing image analysis framework is shown in Figure 11.

The framework in the elasticity mode is tested through a series of asynchronous requests at the scale of tens and hundreds. It is shown that the response time stays below 1.5 s until the concurrency approaches about 700, and then the response time starts to increase linearly. When the asynchronous request is under 200, the maximum execution time of the DAG is less than about 1.5 s. The average response time is still under about 1.5 s though the scale of asynchronous requests reaches 1000 and the maximum execution time is close to 3.5 s. In contrast with the traditional technologies of cloud computing, which may scale in minutes or hours, the serverless-based framework can increase the number of functions and instances in seconds to handle new requests.

7. Discussion and Conclusions

There are some remote sensing frameworks developed from traditional parallel databases and cloud computing technologies. The paper [1] analyzed the related works of three types, including spatial databases, programming and software tools, and big spatial data infrastructures, and categorized them further into ten types from underlying general technologies. Though extensive in the scope of investigation, it only focuses on the underlying technologies without evaluating them in terms of computing service. Different from this review, here, we analyze the representative remote sensing image analysis frameworks from both computing service types and underlying cloud computing technologies.

As one of the most popular remote sensing analysis cloud platforms, GEE provides two types of computing services, namely on-the-fly computing and batch computing. On-the-fly computing is used for rapid prototyping for tiled remote sensing images while batch computing provides the capability of planetary-scale processing. Despite their different focuses, they both have the same programming interface and can switch to each other seamlessly based on GEE’s ability to translate the user-defined codes or functions to DAGs. However, GEE cannot express focal operation with structure locality at a pixel-level. In the study, the ability of the focal operation is implemented through window operators based on relative coordinates and overlapped logical regions.

Different from GEE’s local tiling strategy, the Open Data Cube (ODC) [22] reformats the raw, remote sensing images into analysis-ready data with a global tiling strategy. ODC’s rapid prototyping and parallel computing capabilities are provided by the xarray [23] and Celery framework, respectively. Although Celery has high performance, from the perspective of ease of use, ODC only has a low-level programming model compared with highly customed remote sensing processing APIs or operators and lacks control over operator access. As cloud computing systems are considered to prioritize the ease of use over high performance, we provide end users with a customed DAG-centric programming model that includes various datatypes and operators dedicated to remote sensing data analysis and operator-level access control capabilities.

GeoTrellis [24,25] and GeoSpark [26], built on the top of Spark, are generally used for batch processing of raster data and similar to the batch computing component of GEE, which is based on FlumeJava [27]. These frameworks usually have a distributed data abstraction customed to raster data based on the RDD data structure [28]. However, at the phase of algorithm exploratory, they cannot be used as REPL tools to provide a public computing service and rapid prototyping. Besides, these frameworks require the computation to be built on the distributed datatypes, which limits the expressiveness and flexibility of end-user programming for remote sensing collection processing. Although Spark can be refactored on serverless technologies, GeoTrellis and GeoSpark cannot be directly used as public cloud commodities to provide end-user programming services.

Iris [29] is a python library used for the analysis and visualization of meteorological data, which provides the ability of batch geocomputation based on the distributed numerical computing framework Dask. Contrary to the Spark-based frameworks, Iris has a higher communication efficiency and is capable of performing certain high-performance computing tasks, such as a dense linear algebra calculation. Nevertheless, Iris is highly customed to the analysis of meteorological data, which is usually organized as a NetCDF file or its variants, and, therefore, could not be applied directly to remote sensing images analysis. Besides, as Iris is deeply bound to Dask, it cannot control operator access and provide the service of paying in proportion to resources used, similar to a serverless public commodity.

With the rise of the disaggregation datacenter, serverless computing is believed to become the default computing paradigm of cloud computing and bring closure to the client–server era [8]. Although GEE has the feature of scaling automatically and billing on usage, it does not claim to be built on serverless technologies. All other remote sensing image processing frameworks need explicit resource provisioning, which can be regarded as based on a server-centric computing paradigm. Despite that serverless cloud functions are becoming more and more lightweight and have been successfully employed for several types of general workloads [30,31], there are still many limitations. The cloud functions are stateless without fine-grained coordination and do not provide high-level parallel operators, posing difficulty for remote sensing data processing workloads.

This paper presents the empirical characteristics and a formal definition of on-the-fly cloud geocomputation for the first time. Then, we give a serverless-based software architecture and some proof-of-concept experiments, which suggest that on-the-fly cloud geocomputation can be efficiently implemented with serverless technologies, such as the object storage system and function computing engine. At the frontend, we provide a DAG-based, end-user programming environment for remote sensing data analysis, which contains a series of customed datatypes and operators. The DAG is one of the core designs, which bridges the user-defined code and the cloud functions at the backend. The logical region is another core design, which guarantees the correctness of focal operators through overlapped zones and controls the amount of input image tiles to achieve a rapid response.

Nevertheless, several aspects of the proposed serverless-based system could be further improved. First, the technology stack of current serverless commodities lacks in-memory storage, similar to Redis, which limits further performance improvements. Future work could refer to Anna [32], which can provide high-performance memory storage services to improve the efficiency of COG reading and cache. In addition, the scheduler in the current system adopts a staged scheduling method, which does not pay attention to the difference in execution time between different nodes. Therefore, it is necessary to develop a scheduling algorithm specially oriented to serverless computing to achieve the optimization of both job completion time and cost of execution [33].