Necessary and Sufficient Second-Order Optimality Conditions on Hadamard Manifolds

Abstract

This work is intended to lead a study of necessary and sufficient optimality conditions for scalar optimization problems on Hadamard manifolds. In the context of this geometry, we obtain and present new function types characterized by the property of having all their second-order stationary points be global minimums. In order to do so, we extend the concept convexity in Euclidean space to a more general notion of invexity on Hadamard manifolds. This is done employing notions of second-order directional derivatives, second-order pseudoinvexity functions, and the second-order Karush–Kuhn–Tucker-pseudoinvexity problem. Thus, we prove that every second-order stationary point is a global minimum if and only if the problem is either second-order pseudoinvex or second-order KKT-pseudoinvex depending on whether the problem regards unconstrained or constrained scalar optimization, respectively. This result has not been presented in the literature before. Finally, examples of these new characterizations are provided in the context of “Higgs Boson like” potentials, among others.

1. Introduction

This article concerns itself with several questions, the first of which is why we study the optimality conditions of scalar mathematical programming problems. Optimality conditions are a crucial asset to solve optimization problems, which constitute some of the most ubiquitous types of problems across many scientific disciplines. Furthermore, optimality conditions and their associated optimal points play a vital role in activities as interdisciplinary as finding the best-fit parameters of a model given a set of data. In computational biology, the RNA sequence alignment problem or the evolution of median molecules has been investigated as a single-objective optimization [1].

The second and most important of these questions is the generalization and use of the notion of convexity beyond standard Euclidean geometries. Convexity conditions and their different extensions are best understood by looking at the properties they bring to an optimization problem. Hence, ensuring convexity conditions in an optimization problem guarantees that every found local optimum is indeed a global optimum. Other functions such as pseudoconvexity ensure us that every critical point is also an optimum. However, this later property does not fully characterize pseudoconvex functions as it is also shared with other types of functions. For example, the invex functions, first described by Hanson [2], are defined as any function where critical and optimal points coincide. The equivalence between critical and optimal points makes it possible to design practical numerical methods or algorithms to find solutions to problems that satisfy this condition. Accompanying this, replacing the x-y vector in the definition of the convex function by a certain function or kernel $\eta (x,y)$ makes the invex definition concept much more flexible and applicable to other fields, such as fuzzy or interval-valued environments [3,4].

In scalar mathematical programming problems with constraints, the invexity of the objective function is not sufficient to guarantee that a Karush–Kuhn–Tucker stationary point is an optimum. Thus, it is also necessary to impose conditions on the constraints of the problem. Following these ideas, Martin [5] first defined the KT-invex function. It is also well known that the first-order KKT necessary conditions only provide us with possible candidate solutions, the KKT stationary points, while the second-order necessary conditions discard some of these points to find the optimum. The importance of these second-order conditions must be stressed since optimums cannot be found making exclusive use of the first-order classical optimality conditions. An example of this is illustrated in Ginchev and Ivanov [6].

Until recently, studies have mainly explored these problems on standard Euclidean geometry. However, efforts have shifted towards developing a good understanding of them on different varieties of Riemannian manifolds such as the Hadamard manifolds considered in our study. While the Euclidean space is a linear flat space, Riemannian manifolds are non-linear curved surfaces. In convex optimization, the convexity of a set in a linear space is linked to the possibility of connecting two points of the space through segments of straight lines. Since the role of the straight lines in Euclidean space is carried out by geodesics on manifolds, translating a problem to a non-Euclidean setup partially amounts to replacing the linear segments by geodesic arcs.

Studying optimization problems on Hadamard manifolds poses great advantages since generally, solving nonconvex constrained problems on ${\mathbb{R}}^n$ with a Euclidean metric can be rephrased as solving the unconstrained convex minimization problem on a Hadamard manifold feasible set with an appropriate metric [7]. In this way, we can take advantage of the convexity of the geometry.

Besides the field of mathematics, Hadamard manifolds have attracted a substantial amount of interest. For example, in medicine, Hadamard manifolds are used in medical imaging analysis to quantify tumour growth and consequently infer disease progression. Furthermore, problems related to stereo vision processing [8] or the study of machine learning or computer vision cannot be solved in classical spaces and requires the help of Hadamard manifolds for their modelling [9].

In recent times, several authors have studied these topics. The invexity on Riemannian manifolds and its relationship with the monotonicity were studied by Barani and Pouryayevali [10], and the Mond–Weir dual problems for the optimization problems under invexity assumptions were also obtained on Hadamard manifolds by Zhou and Huang [11]. More closely related to our study, Ginchev and Ivanov [6] obtain second-order optimality conditions of the KKT type for a problem with inequality constraints using pseudoconvex functions and Euclidean spaces. In [12], the author utilized second-order Fréchet differentiable functions in ${\mathbb{R}}^n$, and he defined the second-order KT-pseudoconvexity to prove that each KT point of second-order is a global minimum. Two years later, the same author, in [13], extended some of these results to invex functions, but always in Euclidean spaces.

In Ruiz-Garzón et al. [14], we obtained first-order optimality conditions for the scalar and vector optimization problem on Riemannian manifolds, but not second-order conditions; this will be discussed in this article. Later, in [15], the authors proved the non-smooth version of the previous conditions. Furthermore, in Ruiz-Garzón et al. [16], we proved the existence of optimality conditions from KKT to constrained vector optimization problems on Hadamard manifolds as a particular case of equilibrium vector problems with constraints.

Motivated by Ivanov’s work mentioned previously, the objective of this paper will focus on extending the second-order optimality conditions obtained in Euclidean spaces to geometries beyond ${\mathbb{R}}^n$; in particular, to Hadamard manifolds. Thus, we extend the concept of KT-invexity to Hadamard manifolds. Therefore, we present necessary and sufficient optimality conditions for both unconstrained and constrained scalar optimization problems on Hadamard manifolds, looking for the function types for which the second-order critical points and the global minimum points coincide.

Next, we will give the different sections of this article. In Section 2, we recall concepts related to Hadamard manifolds, and we define the second-order directional derivative function on Hadamard manifolds. In Section 3, we prove that second-order invexity and second-order pseudoinvexity functions coincide and are characterized by the fact that all second-order critical points are equivalent to optimums for unconstrained optimization problem. In Section 4, we extend the previous results to a constrained optimization problem by defining new concepts such as the second-order KKT stationary point and second-order KKT-pseudoinvex problem to characterize the global optimums of these problems on Hadamard spaces. We will end with some conclusions and references.

2. Preliminaries

Let M be an n-dimensional differential manifold. We denote by $T_{x}M$ the n-dimensional tangent space of M at x, which is a real vector space of the same dimension as M. The collection of all tangent spaces of M is denoted by $TM={\bigcup }_{x\in M}{T}_{x}M$, and it is the tangent bundle of M.

Given a piecewise $C^1$ curve $\alpha :[a,b]\to M$, we denote its norm by ${\parallel .\parallel }_x$ and its corresponding length as $L\left(\alpha \right)={\int }_a^b{\parallel {\alpha }^{\prime }\left(t\right)\parallel }_{\alpha \left(t\right)}dt$. For any two points $x,y\in M$, the distance d between the points can be defined as $d(x,y)=\inf \left\{L\left(\alpha \right)\right|\alpha \mathrm{is}\mathrm{a}\mathrm{piecewise}{C}^1\mathrm{curve}\mathrm{joining}x\mathrm{and}y\}$.

Furthermore, any curve $\alpha$ joining x and y in M such that $L\left(\alpha \right)=d(x,y)$ constitutes a geodesic. We will assume that the manifold is complete; in other words, that any two points x and y can be connected with a geodesic whose length is $d(x,y)$.

For differentiable manifolds, it is broadly possible to define the derivatives of the curves on the manifold. The derivative at a point x on the manifold lies in a vector space $T_{x}M$. We define the Riemannian exponential map $\exp :\overline{T}M\to M$ as ${\exp }_x\left(v\right)={\alpha }_v\left(1\right)$ for every $v\in \overline{T}M$, where ${\alpha }_v$ is the geodesic starting at x with velocity v (i.e., $\alpha \left(0\right)=x,{\alpha }^{\prime }\left(0\right)=v$), and $\overline{T}M$ is an open neighbourhood of $TM$. Moreover, ${exp}_{x}tv={\alpha }_v(t,x)$ for each real number t.

With the goal of generalizing the concept of the convex set to Hadamard manifolds, in [17], the author proposed the following definition:

Definition 1.

We will say that a subset $S_1$ of M is totally convex if $S_1$ contains every geodesic ${\alpha }_{x,y}$ of M whose endpoints x and y belong to $S_1$.

It is well known that a simply connected complete Riemannian manifold of non-positive sectional curvature is called a Hadamard manifold. There, the geodesic between any two points is unique. In addition to this, the exponential map at each point of the manifold, M, is a global diffeomorphism. Moreover, this exp map is defined on the whole tangent space and ${exp}_x^{-1}:M\to T_{x}M$.

Example 1.

The space of symmetric $n\times n$ positive-definite matrices $S(n,\mathbb{R})$ endowed with the Frobenius metric defined by $<U,V>=tr(U,V)$ is an example of a Hadamard manifold.

Let $\eta :M\times M\to TM$ be a function defined on the product manifold such that $\eta (x,y)\in T_{y}M,\forall x,y\in M$.

Now, to achieve the objectives of this article, we will need an adequate concept of the differential and second-order differential on Hadamard manifolds:

Definition 2.

Let M be a Hadamard manifold. We define the differential of the function $\theta :M\to \mathbb{R}$ at the point $\overline{x}$ along the direction $\eta (x,\overline{x})$ as $d{\theta }_{\overline{x}}\left(\eta (x,\overline{x})\right)=\langle grad{\theta }_{\overline{x}},\eta (x,\overline{x})\rangle$ for all $\eta (x,\overline{x})$ in $T_{\overline{x}}M$ where $grad{\theta }_{\overline{x}}$ is the gradient of function θ at $\overline{x}$.

Remark 1.

The differential of θ at $\overline{x}$ of $\eta (x,\overline{x})$ is similar to the definition of the directional derivative in the Euclidean space.

If we wish to study the second-order optimality conditions, we need to take a step forward. Thus, we propose the following definition of the second-order directional derivative on Hadamard manifolds:

Definition 3.

Let M be a Hadamard manifold. A mapping $\theta :M\to \mathbb{R}$ is said to be a second-order directional derivative function at the point $\overline{x}$ along the direction $\xi (x,\overline{x})\in T_{\overline{x}}M$ if and only if the limit:

$${\theta }^{″}(\overline{x};\xi (x,\overline{x}))=\underset{t\to 0+}{lim}2t^{-2}\left[\theta \left(ex{p}_{\overline{x}}\left(t\xi (x,\overline{x})\right)\right)-\theta \left(\overline{x}\right)-td{\theta }_{\overline{x}}\left(\xi (x,\overline{x})\right)\right]$$

exists and it is finite. The function θ is called second-order directionally differentiable on M if the second-order directional derivative function exists for each point of M and any direction $\xi (x,\overline{x})\in T_{\overline{x}}M$.

3. Applications to Scalar Optimization Problems

In this section, we will characterize the functions whose critical points are global optimums living on Hadamard’s spaces in the context of scalar optimization problems.

3.1. Unconstrained Case

We start by considering the unconstrained scalar optimization problem (SOP):

$$\begin{array}{cc}\left(\mathrm{SOP}\right)& \min \theta \left(x\right)\\ & \mathrm{subject\; to}& x\in M\end{array}$$

where M is a Hadamard manifold and $\theta :M\to \mathbb{R}$ is a differentiable function.

Let us now introduce the notion of the invexity of a function on Hadamard manifolds, guided by the concept of convexity on linear spaces:

Definition 4.

Let M be a Hadamard manifold. A differentiable function $\theta :M\to \mathbb{R}$ is said to be an invex (IX) at $\overline{x}\in M$ if there exists $\eta (x,\overline{x})\in T_{\overline{x}}M$ such that:

$$\theta \left(x\right)-\theta \left(\overline{x}\right)\ge d{\theta }_{\overline{x}}\left(\eta (x,\overline{x})\right),\forall x\in M.$$

Remark 2.

Note that if $M={\mathbb{R}}^n$, we have the classic and well-known invexity definition given by Hanson [2]. Thus, the previously defined inverse exponential map simplifies to the familiar form ${exp}_x^{-1}y=y-x$.

Once having introduced the concept of invexity, we can know discuss the existence of critical points. For this purpose, we will adopt the definition given by Ruiz-Garzón et al. [14]:

Definition 5.

Let M be a Hadamard manifold. Suppose that the function $\theta :M\to \mathbb{R}$ is differentiable at any $x\in M$. A feasible point $\overline{x}$ for SOP is said to be a critical point (CP) if there exists some $\eta (x,\overline{x})\in T_{\overline{x}}M$ non-identically zero such that $d{\theta }_{\overline{x}}\left(\eta (x,\overline{x})\right)=0$.

Theorem 1.

[14] Let M be a Hadamard manifold. Suppose that the function $\theta :M\to \mathbb{R}$ is differentiable at any $x\in M$. A function is invex on M if and only if every critical point is a global minimum.

$$\boxed{\mathrm{IX}\iff [\mathrm{CP}\iff \mathrm{global} \mathrm{minimum} \mathrm{SOP}]}$$

Remark 3.

This result is a generalization to Riemannian manifolds, a similar result to the one achieved by Craven and Glover [18]. Obviously, if a function has no critical points, so it has no global minimums, then it is invex.

Given these definitions, we are in a position in which we can tackle one of the objectives of our work. Thus, we will now propose a generalization of the concept of the two-invexity given by Ivanov [13] for Fréchet differentiable functions in dimensional finite Euclidean space ${\mathbb{R}}^n$ to Hadamard manifolds M.

Definition 6.

Let M be a Hadamard manifold. A differentiable and the second-order directionally differentiable $\theta :M\to \mathbb{R}$ function are said to be a two-invex (2-IX) at $\overline{x}\in M$ if there exist $\eta (x,\overline{x}),\xi (x,\overline{x})\in T_{\overline{x}}M$ such that the derivative ${\theta }^{″}(\overline{x},\xi (x,\overline{x}))$ exists and:

$$\theta \left(x\right)-\theta \left(\overline{x}\right)\ge d{\theta }_{\overline{x}}\left(\eta (x,\overline{x})\right)+{\theta }^{″}(x,\xi (x,\overline{x})),\forall x\in M.$$

Remark 4.

Note that each convex or invex function is also a two-invex function. This implies that the class of two-invex functions extends the class of invex ones.

Example 2.

Let $M=\{x\in \mathbb{R}:x>0\}$ be endowed with the Riemannian metric $G\left(x\right)=x^{-2}$. Let $\alpha :[0,1]\to M$ be a geodesic defined as $\alpha \left(t\right)=x{e}^{(v/x)t}$, where v is a unit vector. Let ${exp}_x\left(tv\right)=x{e}^{(v/x)t}$ and ${exp}_{\overline{x}}^{-1}x=\overline{x}ln(x/\overline{x})$ be the Riemannian exponential and its inverse map. Let $\theta :M\to \mathbb{R}$ be a function defined as $\theta \left(x\right)={(x-1)}^2$. This function is a two-invex (2-IX) function at $\overline{x}\in M$ because there exist a $\eta (x,\overline{x})=\xi (x,\overline{x})={exp}_{\overline{x}}^{-1}x\in T_{\overline{x}}M$ such that the derivative ${\theta }^{″}(\overline{x},\xi (x,\overline{x}))$ exists and $\theta \left(x\right)-\theta \left(\overline{x}\right)\ge d{\theta }_{\overline{x}}\left(\eta (x,\overline{x})\right)+{\theta }^{″}(x,\xi (x,\overline{x})),\forall x\in M$. Moreover, θ is a convex function, and therefore, it is two-invex.

Furthermore, we can similarly extend the concept of the stationary point:

Definition 7.

Let M be a Hadamard manifold. Suppose that the function $\theta :M\to \mathbb{R}$ is differentiable and second-order directionally differentiable at any $\overline{x}\in M$ and along every direction. A feasible point $\overline{x}$ for SOP is said to be a two-critical point (2-CP), if there exists some $\eta (x,\overline{x})\in T_{\overline{x}}M$ non-identically zero such that $d{\theta }_{\overline{x}}\left(\eta (x,\overline{x})\right)=0$, and if for some direction $\xi (x,\overline{x})\in T_{\overline{x}}M$, there exists the derivative ${\theta }^{″}(\overline{x},\xi (x,\overline{x}))$, then ${\theta }^{″}(\overline{x},\xi (x,\overline{x}))\ge 0$.

Thus, we can propose and prove the following theorem that characterizes the concept of two-invex functions on Hadamard manifolds:

Theorem 2.

Let M be a Hadamard manifold. Let $\theta :M\to \mathbb{R}$ be a differentiable and second-order directionally differentiable function at any $\overline{x}\in M$ along any direction. The function θ is two-invex at every $\overline{x}\in M$ if and only if each 2-CP is a global minimum of θ on M.

Proof. 

We will argue this proof using reductio ad absurdum. Let us begin by making the hypothesis that $\theta$ is two-invex at every $\overline{x}\in M$ and $\overline{x}$ is a 2-CP, but it is not a global minimum. Therefore, there exists a point $x\in M$ such that $\theta \left(x\right)<\theta \left(\overline{x}\right)$. Thus, it follows that $d{\theta }_{\overline{x}}\left(\eta (x,\overline{x})\right)=0$. Moreover,

$${\theta }^{″}(\overline{x},\xi (x,\overline{x}))\ge 0$$

(1)

for some $\xi (x,\overline{x}))\in T_{\overline{x}}M$ such that ${\theta }^{″}(\overline{x},\xi (x,\overline{x}))$ exists.

By the two-invexity of $\theta$, there exist a $\mu (x,\overline{x})\in T_{\overline{x}}M$ and a $\xi (x,\overline{x})\in T_{\overline{x}}M$ such that ${\theta }^{″}(\overline{x},\xi (x,\overline{x}))$ exists and:

$$0>\theta \left(x\right)-\theta \left(\overline{x}\right)\ge d{\theta }_{\overline{x}}\left(\eta (x,\overline{x})\right)+{\theta }^{″}(\overline{x},\xi (x,\overline{x}))={\theta }^{″}(\overline{x},\xi (x,\overline{x}))$$

which is a contradiction with (1).

Now, we will prove the sufficient condition. Suppose each 2-CP is a global minimum; we need to prove that there exist $\eta (x,\overline{x}),\xi (x,\overline{x})\in T_{\overline{x}}M$ such that the derivative ${\theta }^{″}(\overline{x},\xi (x,\overline{x}))$ exists and:

$$\theta \left(x\right)-\theta \left(\overline{x}\right)\ge d{\theta }_{\overline{x}}\left(\eta (x,\overline{x})\right)+{\theta }^{″}(x,\xi (x,\overline{x})),\forall x\in M.$$

(2)

This is ensured by defining, for example, $\eta (x,\overline{x})=tgrad\theta \left(\overline{x}\right)$ where t is an arbitrary positive real and $\xi (x,\overline{x})=0$; then, (2) holds, and $\theta$ is two-invex. □

Therefore, we finally find that:

$$\boxed{2-\mathrm{IX}\iff [2-\mathrm{CP}\iff \mathrm{global} \mathrm{minimum} \mathrm{SOP}]}$$

This result extends Theorem 2.6 given by Ivanov [13] for the finite-dimensional Euclidean space to the Hadamard manifold.

We consider now an example of a possible application of the previous characterization in Hadamard manifolds.

Example 3.

Let us consider the following unconstrained scalar optimization problem:

$$\begin{array}{ccc}\left(SOP\right)& \min \theta \left(p\right)& =\left[\right(\frac{2}{\sqrt{3}{\pi }^{1/4}})(1-p_2^2)exp\left(\frac{-p_2^2}{2}\right)]+1\\ & \mathrm{subject\; to}& p=(p_1,p_2)\in \mathsf{\Omega }.\end{array}$$

The objective function corresponds to the Ricker wavelet, usually referred to as the Mexican hat wavelet (see Figure 1). The Ricker wavelet is a function with many applications within the field of physics such as the modelling of seismic data. Furthermore, this wavelet can be understood as the cross-section of a Higgs–Anderson potential; a potential of great relevance in explaining the inner workings of the Higgs Boson and other modern topics of condensed matter physics. Note that the tails of the original Higgs–Anderson potential diverge to infinity, while in our example, they converge to a constant value.

Moreover, let us further consider the set $\mathsf{\Omega }=\{p=(p_1,p_2)\in {\mathbb{R}}^2:p_2>0\}$. Finally, let G be the Riemannian metric tensor of our Hadamard space, here chosen to be 2 × 2 matrix defined by $G\left(p\right)=\left(g_{ij}\left(p\right)\right)$ with $g_{11}\left(p\right)=g_{22}\left(p\right)=\frac{1}{p_2^2}$, $g_{12}\left(p\right)=g_{21}\left(p\right)=0$.

Endowing $\mathsf{\Omega }$ with the Riemannian metric, we can define the inner product of two vectors, u and v, lived on such a space as $\ll u,v\gg =<G\left(p\right)u,v>$. Furthermore, the gradient is then defined as $grad\theta \left(p\right)=G{\left(p\right)}^{-1}\nabla \theta \left(p\right)$. Thus, we obtain a complete Hadamard manifold ${\mathbb{H}}^2$, representative of the upper half-plane of a hyperbolic space.

Thus, we can prove that the function $\theta$ is two-invex at $\overline{p}$, but not invex. We can show this by considering a $\eta (p,\overline{p})=\xi (p,\overline{p})=3p-\overline{p}$ and $\eta (p,\overline{p})=(0,1)$ such that:

$$d{\theta }_{\overline{p}}\left(\eta (p,\overline{p})\right)=<\left(p_2^2\left(0,\left(\frac{2}{\sqrt{3}{\pi }^{1/4}}\right)p_2(p_2^2-3)exp\left(\frac{-p_2^2}{2}\right)\right)\right),\left(\begin{array}{c}0\\ 1\end{array}\right)>$$

$$d{\theta }_{\overline{p}}\left(\eta (p,\overline{p})\right)=p_2^2\left(\frac{2}{\sqrt{3}{\pi }^{1/4}}\right)p_2(p_2^2-3)exp\left(\frac{-p_2^2}{2}\right).$$

On the one hand, $\overline{p}=(0,1.87)$ is a critical point and a global maximum. On the other hand, $\overline{p}=(-\sqrt{3},0.61)$ and $\overline{p}=(\sqrt{3},0.61)$ are two 2-CP and global minimums. By the previous Theorem 2, $\theta$ is a second-order invex (2-IX) function since the sets of two-stationary points (2-CP) and global minimums coincide. However, the objective function is not invex (IX) because the set of CP points does not coincide with global minimums. Therefore, from Examples (2) and (3):

$$\begin{array}{ccc}\boxed{\left(\mathrm{IX}\right)}& \Rightarrow & \boxed{(2-\mathrm{IX})}\\ & \nLeftarrow \end{array}$$

In generalized convexity theory, it is well known that pseudoconvex functions are a generalization of convex functions in Euclidean spaces, and they ensure us that all critical points are optimal. Now, in the same way, we will concern ourselves with extending the concept of the two-invexity function to the two-pseudoinvex function on Hadamard manifolds.

Definition 8.

Let M be a Hadamard manifold. Moreover, let $\theta :M\to \mathbb{R}$ be a differentiable and second-order directionally differentiable function at any $\overline{x}\in M$ and along every direction. A differentiable θ function is said to be a two-pseudoinvex (2-PIX) at $\overline{x}\in M$ if there exist $\eta (x,\overline{x}),\xi (x,\overline{x})\in T_{\overline{x}}M$ such that:

$$\theta \left(x\right)-\theta \left(\overline{x}\right)<0\Rightarrow \left\{\begin{array}{c}d{\theta }_{\overline{x}}\left(\eta (x,\overline{x})\right)\le 0\hfill \\ d{\theta }_{\overline{x}}\left(\eta (x,\overline{x})\right)=0\Rightarrow {\theta }^{″}(x,\xi (x,\overline{x}))<0\hfill \end{array}\right.$$

for all $x\in M$.

We will now analyse the relationship between 2-IX and 2-PIX functions.

Theorem 3.

Let M be a Hadamard manifold. Let $\theta :M\to \mathbb{R}$ be differentiable and second-order directionally differentiable at every $\overline{x}\in M$ and along every direction such that $x\in M$, $\theta \left(x\right)<\theta \left(\overline{x}\right),d{\theta }_{\overline{x}}\left(\eta (x,\overline{x})\right)=0$. If θ is a two-pseudoinvex function at $\overline{x}\in M$, then θ is also two-invex at $\overline{x}\in M$.

Proof. 

In order to prove this, let us make the following assumption. Let $\overline{x},x\in M$ be two points such that $\theta \left(x\right)<\theta \left(\overline{x}\right)$.

(a)

If $d{\theta }_{\overline{x}}\left(\eta (x,\overline{x})\right)<0$, then the inequality:

$$d{\theta }_{\overline{x}}\left(\eta (x,\overline{x})\right)+{\theta }^{″}(x,\xi (x.\overline{x}))<0$$

(3)

is ensured with $\xi (x,\overline{x})=0$.

(b)

If $d{\theta }_{\overline{x}}\left(\eta (x,\overline{x})\right)=0$, then ${\theta }^{″}(x,\xi (x,\overline{x}))$ exists and Inequality (3) holds since $\theta$ is two-pseudoinvex.

The inequality (3) implies the two-invexity of $\theta$ since if $\theta$ is not a two-invex function, then there exist $\eta (x,\overline{x}),\xi (x,\overline{x})\in T_{\overline{x}}M$ such that the derivative ${\theta }^{″}(\overline{x},\xi (x,\overline{x}))$ exists and $\theta \left(x\right)-\theta \left(\overline{x}\right)<d{\theta }_{\overline{x}}\left(\eta (x,\overline{x})\right)+{\theta }^{″}(x,\xi (x,\overline{x}))<0$, which is incompatible with all $\overline{x}$ being a minimum. That is, the 2-PIX implies 2-IX. □

We can even go one step further very easily:

Theorem 4.

Let M be a Hadamard manifold. Let $\theta :M\to \mathbb{R}$ be differential and second-order directionally differential at every $\overline{x}\in M$ in every direction such that $x\in M$, $\theta \left(x\right)<\theta \left(\overline{x}\right),d{\theta }_{\overline{x}}\left(\eta (x,\overline{x})\right)=0$. The function θ is two-pseudoinvex at $\overline{x}\in M$ if and only if θ is two-invex at $\overline{x}\in M$.

Proof. 

On the one hand, from Theorem 3, the two-pseudoinvexity implies the two-invexity. On the other hand, it is well known that the two-invexity implies the two-pseudoinvexity. □

Joining the theorems 2 and 4, we get the following conclusion:

Corollary 1.

Let M be a Hadamard manifold. Let $\theta :M\to \mathbb{R}$ be differential and second-order directionally differential at every $\overline{x}\in M$ in every direction such that $x\in M$, $\theta \left(x\right)<\theta \left(\overline{x}\right),d{\theta }_{\overline{x}}\left(\eta (x,\overline{x})\right)=0$. The function θ is two-pseudoinvex at $\overline{x}\in M$ if and only if each 2-CP is a global minimum of θ on M.

In summary, we have that:

$$\boxed{2-\mathrm{PIX}\iff [2-\mathrm{CP}\iff \mathrm{global} \mathrm{minimum} \mathrm{SOP}]}$$

The previous results extend the results obtained by Ivanov, Theorems 2.12 and 2.14 in [13], from an environment of convexity to a more general environment of invexity. Furthermore, we generalize these notions from Euclidean space to Hadamard manifolds. Therefore, the two-pseudoinvexity coincides with the two-invexity in the same way that the pseudoinvexity coincides with the invexity, as demonstrated by Craven and Glover in [18] on Euclidean space.

3.2. Constrained Case

We consider the constrained scalar optimization problem of the form:

$$\begin{array}{cc}\left(\mathrm{CSOP}\right)& \min \theta \left(x\right)\\ & \mathrm{subject\; to}& g_j\left(x\right)\le 0,j=1,2,\dots ,m,x\in M.\end{array}$$

where M is Hadamard manifold and $\theta ,g_j:M\to \mathbb{R},j=1,2,\dots ,m$ is a set of differentiable functions. Let us consider $S_1=\{x\in M|g_j\left(x\right)\le 0,j=1,2,\dots ,m\}$, and let $I\left(x\right)$ be the set of active constraints. The equality constraints $h_j\left(x\right)$ can be considered inequality constraints as $h_j\left(x\right)\le 0$ and $-h_j\left(x\right)\le 0$.

Similarly to the unconstrained case, our aim is to find the kind of functions lived on Hadamard spaces for which the Karush–Kuhn–Tucker points and the optimums coincide. For this purpose, let us consider quasiinvex functions. These functions are a generalization of quasiconvex functions, a type of function that shares most of its properties with convex and pseudoconvex functions. However, as opposed to pseudoconvex functions, the critical points of quasiconvex functions may not be optimal.

Definition 9.

Let M be a Hadamard manifold. Let $\theta :M\to \mathbb{R}$ be a differentiable function. Then, θ is said to be quasiinvex at $\overline{x}\in M$ if there exist $\eta (x,\overline{x})\in T_{\overline{x}}M$ such that $\theta \left(x\right)\le \theta \left(\overline{x}\right)\Rightarrow d{\theta }_{\overline{x}}\left(\eta (x,\overline{x})\right)\le 0$.

We employed the following definition of critical direction:

Definition 10.

Let M be a Hadamard manifold. A direction $\eta (x,\overline{x})$ is called critical at the point $\overline{x}\in S_1\subseteq M$ if:

$$d{\theta }_{\overline{x}}\left(\eta (x,\overline{x})\right)\le 0andd{g}_{j\left(\overline{x}\right)}\left(\eta (x,\overline{x})\right)\le 0\forall j\in I\left(\overline{x}\right).$$

In the constrained case, the concept of critical point we explored in the unconstrained case is replaced by this concept.

Definition 11.

Let M be a Hadamard manifold. Suppose that the functions $\theta ,g_j:S_1\subseteq M\to \mathbb{R},j=1,2,\dots ,m$ are differentiable and second-order directionally differentiable at any $\overline{x}\in S_1$ in every critical direction $\eta (x,\overline{x})\in T_{\overline{x}}M$. A feasible point $\overline{x}$ for constrained scalar optimization problem (CSOP) is said to be a two-Karush–Kuhn–Tucker stationary point (2-KKT point), if for every critical direction $\eta (x,\overline{x})$ non-identically zero, there exist non-negative multipliers $\lambda ,{\mu }_1,\dots ,{\mu }_m$ with $(\lambda ,\mu )\ne (0,0)$ such that:

$$grad{L}_{\overline{x}}=0$$

(4)

$${\mu }_j{g}_j\left(\overline{x}\right)=0,j=1,2,\dots ,m$$

(5)

$$\lambda d{\theta }_{\overline{x}}\left(\eta (x,\overline{x})\right)=0;{\mu }_{j}d{g}_{i\left(\overline{x}\right)}\left(\eta (x,\overline{x})\right)=0,j\in I\left(\overline{x}\right)$$

(6)

$$L^{″}(\overline{x};\eta (x,\overline{x}))=\lambda {\theta }^{″}(\overline{x};\eta (x,\overline{x}))+\sum _{j\in I\left(\overline{x}\right)}{\mu }_j{g}_j^{″}(\overline{x};\eta (x,\overline{x}))\ge 0$$

(7)

where $L=\lambda \theta +{\sum }_{j=1}^n{\mu }_j{g}_j$ is the Lagrange function.

Note that the last two conditions have been added to the classic KKT conditions.

Now, we need to introduce some new concepts that allow us to relate stationary and optimal points in the constrained scalar optimization problem (CSOP), since the invexity of the objective function by itself does not guarantee the identification of stationary and optimal points. Thus, our intention is to extend the kind of KT-invex functions created by Martin [5] to generalized invexity on Hadamard manifolds. In order to do so, let us set the following definitions:

Definition 12.

Let M be a Hadamard manifold. A CSOP problem is said to be 2-KKT-pseudoinvex (2-KKT-PIX) for CSOP, if $\theta \left(x\right)-\theta \left(\overline{x}\right)<0$, then there exists $\eta :M\times M\to TM$ such that for all feasible points x, $\overline{x}$:

$$d{\theta }_{\overline{x}}\left(\eta (x,\overline{x})\right)\le 0$$

(8)

$$d{\theta }_{\overline{x}}\left(\eta (x,\overline{x})\right)=0\Rightarrow {\theta }^{″}(\overline{x},\eta (x,\overline{x}))<0$$

(9)

$$d{g}_{j\left(\overline{x}\right)}\left(\eta (x,\overline{x})\right)\le 0,\forall j\in I\left(\overline{x}\right)$$

(10)

$$d{g}_{j\left(\overline{x}\right)}\left(\eta (x,\overline{x})\right)=0,\forall j\in I\left(\overline{x}\right)\Rightarrow g_j^{″}(\overline{x},\eta (x,\overline{x}))\le 0$$

(11)

where $I\left(\overline{x}\right)=\{j=1,\cdots ,m:g_j\left(\overline{x}\right)=0\}$.

We now can obtain the sufficient condition for global optimality:

Theorem 5.

Let M be a Hadamard manifold. Suppose that the functions $\theta ,g_j:S_1\subseteq M\to \mathbb{R},j=1,2,\dots ,m$ are differentiable and second-order directionally differentiable at any $\overline{x}\in S_1$ in every critical direction. Furthermore, assume that the CSOP is a 2-KKT pseudoinvex problem. Then, each 2-KKT point is a global minimum.

Proof. 

Suppose by hypothesis that the CSOP is a 2-KKT pseudoinvex problem and that $\overline{x}\in S_1$ is a 2-KKT point. In order to prove that $\overline{x}$ is a global minimum, let us assume the opposite and, thus, that there is a $x\in S_1$ with $\theta \left(x\right)<\theta \left(\overline{x}\right)$. Thanks to the 2-KKT-pseudoinvexity, we have that $d{\theta }_{\overline{x}}\left(\eta (x,\overline{x})\right)\le 0$ and $d{g}_{j\left(\overline{x}\right)}\left(\eta (x,\overline{x})\right)\le 0,j\in I\left(\overline{x}\right)$; the direction $\eta (x,\overline{x})$ is critical.

Since x is a 2-KKT stationary point, there exist a $\lambda >0$ and a ${\mu }_j\ge 0,j\in I\left(\overline{x}\right)$ such that expressions (4)–(7) hold. Then, we conclude from $grad{L}_{\overline{x}}=0$ that: $d{\theta }_{\overline{x}}\left(\eta (x,\overline{x})\right)=d{g}_{j\left(\overline{x}\right)}\left(\eta (x,\overline{x})\right)=0,\forall i\in I\left(\overline{x}\right)$ such that ${\mu }_j>0$.

From the 2-KKT-pseudoinvexity of CSOP, we have that ${\theta }^{″}(\overline{x};\eta (x,\overline{x}))<0$ and $g_j^{″}(\overline{x};\eta (x,\overline{x}))\le 0$ for all $j\in I\left(\overline{x}\right)$ with ${\mu }_j\ge 0$. Thus, on the one hand, we get $L^{″}(\overline{x};\eta (x,\overline{x}))<0$, and on the other, hand we get Expression (7), a contradiction. □

Now, we will obtain a necessary condition for optimality.

Theorem 6.

Let M be a Hadamard manifold. Suppose that the functions $\theta ,g_j:S_1\subseteq M\to \mathbb{R},j=1,2,\dots ,m$ are differentiable and second-order directionally differentiable at any $\overline{x}\in S_1$ in every critical direction $\eta (x,\overline{x})\in T_{\overline{x}}M$ and the functions $\theta ,g_j:S_1\subseteq M\to \mathbb{R},j=1,2,\dots ,m$ are quasiinvex differentiable at $\overline{x}\in S_1$ with respect to η non-identically zero. If each 2-KKT point is a global minimum, then the problem CSOP is 2-KKT pseudoinvex.

Proof. 

Given that each 2-KKT stationary point is a global minimum, we will prove that CSOP is 2-KKT pseudoinvex. Given two $x,\overline{x}\in S_1$ points with:

$$\theta \left(x\right)<\theta \left(\overline{x}\right).$$

(12)

The relationships (8)–(11) should be checked:

• Step 1. According to the quasiinvexity of $\theta$ at $\overline{x}\in S_1$, then $d{\theta }_{\overline{x}}\left(\eta (x,\overline{x})\right)\le 0$ holds, and therefore, Expression (8) is verified.
• Step 2. If $d{\theta }_{\overline{x}}\left(\eta (x,\overline{x})\right)=0$, since $\eta (x,\overline{x})$ is non-identically zero, we can prove that ${\theta }^{″}(\overline{x},\eta (x,\overline{x}))<0$. By reductio ad absurdum, suppose that ${\theta }^{″}(\overline{x};\eta (x,\overline{x}))\ge 0$ for all critical directions, then $\overline{x}$ is a 2-KKT point, which implies, by the hypothesis, that $\overline{x}$ is a global minimum, which is in contradiction with Expression (12). Therefore, there exists a critical direction $\eta (x,\overline{x})$ such that ${\theta }^{″}(\overline{x};\eta (x,\overline{x}))<0$; the expression (9) holds.
• Step 3. We need to prove that $d{g}_{j\left(\overline{x}\right)}\left(\eta (x,\overline{x})\right)\le 0$. However, this is a consequence of quasiinvexity, then Expression (10) holds.
• Step 4. The expression:

  $$d{g}_{j\left(\overline{x}\right)}\left(\eta (x,\overline{x})\right)=0,\forall j\in I\left(\overline{x}\right)\Rightarrow g_j^{″}(\overline{x},\eta (x,\overline{x}))\le 0.$$

However, it is follows directly from the assumption $x\in S_1,j\in I\left(\overline{x}\right)$ and the quasiinvexity of $g_j$ at $\overline{x}\in S_1$. Therefore, Expression (11) happens.

In conclusion, all this shows that Equations (8)–(11) hold, and then, the problem CSOP is 2-KKT pseudoinvex. □

Hence, we arrive at the most important outcome of this section that we present in the following corollary:

Corollary 2.

Let M be a Hadamard manifold. Suppose that the functions$\theta ,g_j:S_1\subseteq M\to \mathbb{R},j=1,2,\dots ,m$are differentiable and second-order directionally differentiable at any$\overline{x}\in S_1$in every critical direction$\eta (x,\overline{x})\in T_{\overline{x}}M$and the functions$\theta ,g_j:S_1\subseteq M\to \mathbb{R},j=1,2,\dots ,m$are quasiinvex differentiable at$\overline{x}\in S_1$with respect to η non-identically zero. Then, each 2-KKT point is a global minimum if and only if the problem CSOP is 2-KKT pseudoinvex.

Therefore, we have that under a quasiinvexity environment:

$$\boxed{2-\mathrm{KKT}-\mathrm{PIX}\iff [2-\mathrm{KKT}-\mathrm{point}\iff \mathrm{global} \mathrm{minimum} \mathrm{CSOP}]}$$

This result generalizes Ivanov’s Theorems 3.1 and 3.2 [12] to Hadamard manifolds and to invex environments. Moreover, it also completes with second-order conditions the first-order results obtained by Ruiz-Garzón et. al. [14] for scalar constrained optimization problems.

We now illustrate the previous corollary 2 with an example:

Example 4.

Let us recover the set Ω and the Riemannian metric from our previous example 3. Now, let the (CSOP) be defined as:

$$\begin{array}{cc}\left(\mathrm{CSOP}\right)& \min \theta \left(p\right)=p_1\\ & \mathrm{subject\; to}& g_1\left(p\right)=2p_1-2\ge 0\\ & & g_2\left(p\right)=p_2-1\ge 0.\end{array}$$

Since $\theta ,g_j$ are linear functions $j=1,2,\dots ,m$, therefore they are quasiinvex at $\overline{p}=(1,1)$. Furthermore, there exist $\eta (p,\overline{p})=3p-\overline{p}$ and $\eta (p,\overline{p})=(0,1)$ such that:

$$d{\theta }_{\overline{p}}\left(\eta (p,\overline{p})\right)=<\left(p_2^2,0\right),\left(\begin{array}{c}0\\ 1\end{array}\right)>=0$$

$$d{g}_{\left(\overline{p}\right)}\left(\eta (p,\overline{p})\right)=<\left(\begin{array}{cc}{p}_2^2& 0\\ 0& p_2^2\end{array}\right)\left(\begin{array}{cc}2& 0\\ 0& 1\end{array}\right),\left(\begin{array}{c}0\\ 1\end{array}\right)>=\left(\begin{array}{c}0\\ p_2^2\end{array}\right).$$

Therefore, imposing the values for $\lambda$ and $\mu$, $\lambda =1$ and $\mu =(1,0)$, we obtain that the Equations (4)–(7) hold. In conclusion, $\overline{p}$ is a 2-KKT-point and a global minimum to the (CSOP). According to the previous corollary 2, the (CSOP) is then 2-KKT-PIX.

4. Conclusions

In conclusion, at the beginning of this article, we set ourselves the goal of obtaining second-order characterizations of optimality on Hadamard manifolds for both the unconstrained and constrained scalar optimization problem. This was done by extending the notions of Euclidean environments given in the literature by Ivanov [12,13] to Hadamard manifolds. In order to do so, our work proposed and made use of:

• An adequate second-order differential definition on Hadamard manifolds.
• An extension of the concept of two-invexity given by Ivanov [13] for Fréchet differentiable functions in dimensional finite Euclidean space ${\mathbb{R}}^n$ to Hadamard manifolds.
• An adapted definition of the two-pseudoinvex functions from the classic pseudoconvex functions to this new context.
• Identifying the two-invex with the two-pseudoinvex functions and characterizing them according to the equivalence of second-order critical points and optimal points.
• A definition of the 2-KKT point and critical direction concepts on a Hadamard manifold.
• A definition of the 2-KKT pseudoinvex problems with which to obtain second-order optimality conditions on the constrained scalar optimization problem.

One possible way forward may be related to the extension of these methods to the computational environment, i.e., to obtaining optimal points in Hadamard manifolds.