Some Matrix-Variate Models Applicable in Different Areas

Abstract

Matrix-variate Gaussian-type or Wishart-type distributions in the real domain are widely used in the literature. When the exponential trace has an arbitrary power and when the factors involving a determinant and a trace enter into the model or a matrix-variate gamma-type or Wishart-type model with an exponential trace having an arbitrary power, they are extremely difficult to handle. One such model with factors involving a trace and a determinant and the exponential trace having an arbitrary power, in the real domain, is known in the literature as the Kotz model. No explicit evaluation of the normalizing constant in the Kotz model seems to be available. The normalizing constant that is widely used in the literature, is interpreted as the normalizing constant in the general model, and that is referred to as a Kotz model does not seem to be correct. One of the main contributions in this paper is the introduction of matrix-variate distributions in the real and complex domains belonging to the Gaussian-type, gamma-type, and type 1 and type 2 beta-types, or Mathai’s pathway family, when the exponential trace has an arbitrary power and explicit evaluations of the normalizing constants therein. All of these models are believed to be new. Another new contribution is the logistic-based extensions of the models in the real and complex domains, with the exponential trace having an arbitrary exponent and connecting to extended zeta functions introduced by this author recently. The techniques and steps used at various stages in this paper will be highly useful for people working in multivariate statistical analysis, as well as for people applying such models in engineering problems, communication theory, quantum physics, and related areas, apart from statistical applications.

1. Introduction

Matrix-variate statistical distributions are widely used in all types of disciplines such as statistics, physics, communication theory, and engineering problems. A matrix-variate density where a trace with an exponent enters into the density as a product and when the exponential trace has an arbitrary power, known as the Kotz model in the literature, is widely used in the analysis of data coming from various areas such as multilook return signals in radar and sonar; see, for example, ref. [1] regarding the analysis of PolSAR (polarimetric synthetic aperture radar) data. The Kotz model is a generalization of the basic matrix-variate Gaussian model, or it can also be considered as a generalization of the matrix-variate gamma model or Wishart model, which can be considered as limiting forms of Mathai’s pathway models.

When analyzing radar data, it has been found that Gaussian-based models fit well when the surface is disturbance-free [1]. It has been found that Gaussian-based models are not appropriate in certain regions such as urban areas, sea surfaces, forests, etc.; see, for example [2,3,4,5]. Hence, we will consider some nonGaussian or nonWishart models as well in this paper, along with Gaussian-based models.

In most of the applications in engineering areas, each scalar variable has two components, such as time and phase, so that a complex variable is very appropriate to represent such a scalar variable. Hence, it has been found that the distributions in the complex domain are more important in their applications in physical science and engineering areas [1]. When a statistical density is used in any applied problem, the computation of the normalizing constant there is the most important step, because when studying all sorts of properties of such a model, the computations naturally follow the format of the evaluation of the normalizing constant in that model. The explicit evaluation of the normalizing constant in a general model, often referred to also as a Kotz model in the real domain, does not seem to be available in the literature. The normalizing constant in the general model in the real domain appearing in [6], which the authors claim to have been available elsewhere in the earlier literature, seems to be the one that is widely used in all of the applications where the Kotz model in the real domain is used. However, unfortunately the normalizing constant quoted in [6] does not seem to be correct. A Kotz-type model in the complex domain does not seem to be available in the literature, and the normalizing constant therein does not seem to be available either. Hence, one of the aims of this paper is to give the derivation of the normalizing constant in the general model in detail, in the real and complex domains, and also to extend the ideas to Mathai’s pathway family, namely, the matrix-variate gamma, and the type 1 beta and type 2 beta families of densities. Since the derivation of the normalizing constant is the most important step in the construction of any statistical model, various matrix-variate models are listed in this paper by showing the computations of the normalizing constants in each case. Some applications of the Kotz model in the real domain may be seen in [7,8,9,10].

All of the models in the complex domain discussed in this paper, where the exponential trace has an arbitrary power and the evaluations of the normalizing constants therein, are believed to be new. All of the normalizing constants in the models in the real domain, where the exponential trace has arbitrary power, are also believed to be new. The challenging problem in the evaluation of the normalizing constants is when the function has a determinant and a trace as multiplicative factors, and the exponential trace has an arbitrary power. If the determinant factor is not there, then one can use Lemma 3 given in Section 2 and evaluate the integrals concerned. When the determinant and trace, especially the determinant, appear as multiplicative factors, and, at the same time, the exponential trace has an arbitrary power, then the only way to tackle this scenario that this author can think of is to apply Lemma 4 first and then apply a general polar coordinate transformation. Still, there will be a problem of integrating the sine and cosine factors coming from the the determinant factor. This is solved in Theorems 1 and 2 given in Section 2. This is how the problem is solved, and the above applications are some of the novelties in the present paper. With the current paper, one can assume that the evaluation of the exact normalizing constants in the most generalized rectangular matrix-variate gamma-type, type 1 beta-, and type 2 beta-type models, where the factors involving arbitrary powers for traces and determinants, and an exponential trace having arbitrary powers in gamma-type models appear, are fully solved.

This paper is organized as follows: Section 1 contains the introductory material. Section 2 starts with a subsection giving the notations and terminologies used in this paper, where most of the notations are standard notations used in mathematical and statistical literature, and the remaining ones are introduced in order to avoid a multiplicity of symbols and equation numbers. Then, Section 2 gives explicit evaluation of the normalizing constant in an extended matrix-variate gamma-type, Gaussian-type, Wishart-type, or Kotz-type model, both in the real and complex domains, and then deals with multivariate and matrix-variate extended Gaussian- and gamma-type distributions. Section 3 examines extended matrix-variate type 2 beta-type models in the real and complex domains. Section 4 contains extended matrix-variate models of the type 1 beta type in the real and complex domains. Throughout the paper, the results in the complex domain are listed side by side with the corresponding results in the real domain. Detailed derivations are carried out for the real domain cases only, since most of the steps in the complex domain are parallel to those in the real domain.

2. Evaluation of Some Matrix-Variate Integrals and the Resulting Models

2.1. Notations and Formats

This paper deals with scalar/vector/matrix variables; scalar and matrix constants; square and rectangular matrices; singular, nonsingular, and positive definite matrices; and all are defined in the real and complex domains. Hence, a multiplicity of symbols and equations are needed to represent all of these items distinctly. In order to simplify matters and to bring the function and equation numbers to a manageable stage, the following notations will be used in this paper.

Real scalar variables, whether mathematical or random, will be denoted by lowercase letters such as x and y. Real vector/matrix variables, whether mathematical or random, whether square matrices or rectangular matrices, will be denoted by capital letters such as X and Y. Scalar constants will be denoted by lowercase letters such as a and b, and vector/matrix constants will be denoted by $A,B$, etc. A tilde will be used to designate variables in the complex domain such as $\tilde{x},\tilde{y},\tilde{X},$ and $\tilde{Y}$. No tilde will be used on constants. When Greek letters and other symbols appear, the notations will be explained then and there. Let $X=(x_{ij})$ be a $p\times q$ matrix where the elements are functionally independent or distinct real scalar variables. Then, the wedge product of the differentials will be defined as $\mathrm{d}X={\wedge }_{i=1}^p{\wedge }_{j=1}^q\mathrm{d}{x}_{ij}$. When x and y are real scalars, then the wedge product of their differentials is defined as $\mathrm{d}x\wedge \mathrm{d}y=-\mathrm{d}y\wedge \mathrm{d}x$ so that $\mathrm{d}x\wedge \mathrm{d}x=0,\mathrm{d}y\wedge \mathrm{d}y=0$. For a square matrix A, the determinant will be denoted as $|A|$ or as $\det (A)$. When A is in the complex domain, then the absolute value of the determinant or modulus of the determinant will be denoted as $|\det (A)|$. If $|A|=a+ib,i=\sqrt{(-1)},a,$ and b are real scalars, then $|\det (A)|=\sqrt{(a^2+b^2)}$. If $\tilde{X}$ is in the complex domain, then one can write $\tilde{X}=X_1+i{X}_2,i=\sqrt{(-1)},X_1,$ and $X_2$ as real, and the wedge product of the differentials in $\tilde{X}$ will be defined as $\mathrm{d}\tilde{X}=\mathrm{d}{X}_1\wedge \mathrm{d}{X}_2$. We will consider only real-valued scalar functions in this paper. ${\int }_{X}f(X)\mathrm{d}X$ will denote the integral over X of the real-valued scalar function $f(X)$ of X. When $f(X)$ is a real-valued scalar function of X, whether X is a scalar variable, vector, or matrix in the real or complex domain, and if $f(X)\ge 0$ for all X, and ${\int }_{X}f(X)\mathrm{d}X=1$, then $f(X)$ will be defined as a density or statistical density. When a square matrix X is positive definite, then it will be denoted as $X>O$, where $X=X^{\prime }$, and a prime denotes the transpose. The conjugate transpose of any matrix $\tilde{Y}$ in the complex domain will be written as ${\tilde{Y}}^{*}$. When a square matrix $\tilde{X}$ is in the complex domain, and if $\tilde{X}={\tilde{X}}^{*}$, then $\tilde{X}$ is Hermitian. If $\tilde{X}={\tilde{X}}^{*}>O$, then $\tilde{X}$ is called Hermitian positive definite. When $Y>O$, then ${\int }_A^{B}f(X)\mathrm{d}X$ means the integral of the real-valued scalar function $f(X)$ over the real positive definite matrix X such that $X>O,A>O,B>O,X-A>O,$ and $B-X>O$ (all positive definites), where $A>O$ and $B>O$ are constant matrices, and they have a similar notation and interpretation in the complex domain as well. For example, $B-X>O$ means B is positive definite, X is positive definite, and $B-X$ is positive definite, and the right side of the inequalities is not zero but the capital letter O. In order to avoid a multiplicity of numbers, the following procedure will be used. For a function number in the complex domain, corresponding to the same number in the real domain, a letter c will be affixed to the function number. For example, $f_{1c}(\tilde{X})$ will correspond to $f_1(X)$ in the real domain. This notation will enable a reader to recognize a function in the complex domain instantly by recognizing the subscript c. Other notations will be explained whenever they occur for the first time.

2.2. Some Matrix-Variate Integrals

Let us start with an example of the evaluation of an integral in the real domain, which will show the different types of hurdles to overcome to achieve the final result. Let $X=(x_{ij})$ be a $p\times q,p\le q$ matrix of rank p, where the $pq$ elements $x_{ij}$ are functionally independent (distinct) real scalar variables. Suppose that we wish to evaluate the following integral, where $f(X)$ is a real-valued scalar function of the $p\times q$ matrix X, observing that integral over X and the wedge product of the differentials, $\mathrm{d}X$, are already explained in Section 2.1:

$$\begin{array}{c}{\int }_{X}f(X)\mathrm{d}X\hfill \\ =c{\int }_X{|A^{\frac{1}{2}}(X-M)B{(X-M)}^{\prime }{A}^{\frac{1}{2}}|}^{\gamma }{[\mathrm{tr}(A^{\frac{1}{2}}(X-M)B{(X-M)}^{\prime }{A}^{\frac{1}{2}})]}^{\eta }\hfill \\ \times {\mathrm{e}}^{-\alpha {[\mathrm{tr}(A^{\frac{1}{2}}(X-M)B{(X-M)}^{\prime }{A}^{\frac{1}{2}})]}^{\delta }}\mathrm{d}X\hfill \end{array}$$

(1)

where $\delta >0,M=E[X],\Re (\eta )>0,\Re (\gamma )>-\frac{q}{2}+\frac{p-1}{2},A>O$ is a $p\times p$ positive definite constant matrix, and $B>O$ is a $q\times q$ positive definite constant matrix; $A^{\frac{1}{2}}$ is the positive definite square root of the positive definite matrix $A>O$, $E[·]$ denotes the expected value of $[·]$, and $\Re (·)$ means the real part of $(·)$. The first step here is to simplify the matrix $A^{\frac{1}{2}}(X-M)B{(X-M)}^{\prime }{A}^{\frac{1}{2}}$ into a convenient form by making a transformation of $Y=A^{\frac{1}{2}}(X-M)B^{\frac{1}{2}}\Rightarrow \mathrm{d}Y={|A|}^{\frac{q}{2}}{|B|}^{\frac{p}{2}}\mathrm{d}X$ from the Lemma 1 given below by observing that $\mathrm{d}(X-M)=\mathrm{d}X$, since M is a constant matrix. The corresponding integral in the complex domain is the following:

$$\begin{array}{c}{\int }_{\tilde{X}}{f}_c(\tilde{X})\mathrm{d}\tilde{X}\hfill \\ =\tilde{c}{\int }_X{|\det (A^{\frac{1}{2}}(\tilde{X}-\tilde{M})B{(\tilde{X}-\tilde{M})}^{*})|}^{\gamma }{[\mathrm{tr}(A^{\frac{1}{2}}(\tilde{X}-\tilde{M})B{(\tilde{X}-\tilde{M})}^{*}{A}^{\frac{1}{2}})]}^{\eta }\hfill \\ \times {\mathrm{e}}^{-\alpha {[\mathrm{tr}(A^{\frac{1}{2}}(\tilde{X}-\tilde{M})B{(\tilde{X}-\tilde{M})}^{*}{A}^{\frac{1}{2}})]}^{\delta }}\mathrm{d}\tilde{X}\hfill \end{array}$$

(2)

where $A=A^{*}>O,B=B^{*}>O$ (both Hermitian positive definites), A is $p\times p$, B is $q\times q$, and $\tilde{M}=\tilde{X}$. The transformation in the complex case is $\tilde{Y}=A^{\frac{1}{2}}(\tilde{X}-\tilde{M})B^{\frac{1}{2}}\Rightarrow \mathrm{d}\tilde{Y}=|\det (A){|}^q{|\det (B)|}^p\mathrm{d}\tilde{X}$.

Proofs of the following lemmas are given in detail in [11]. For the sake of illustration, the proof of the real part of Lemma 1 is detailed in Appendix A at the end of this paper.

Lemma 1.

Let the $m\times n$ matrix $X=(x_{ij})$ be in the real domain, where the $mn$ elements $x_{ij}$ are functionally independent (distinct) real scalar variables, and let A be a $m\times m$ nonsingular constant matrix and B be a $n\times n$ nonsingular constant matrix. Then, we have the following:

$$Y=AXB, |A|\ne 0, |B|\ne 0\Rightarrow \mathrm{d}Y={|A|}^n{|B|}^m\mathrm{d}X.$$

Let the $m\times n$ matrix $\tilde{X}$ be in the complex domain, where A and B are $m\times m$ and $n\times n$ nonsingular constant matrices, respectively, in the real or complex domain; then, we have the following:

$$\begin{array}{cc}\hfill \tilde{Y}& =A\tilde{X}B, |A|\ne 0, |B|\ne 0\Rightarrow \hfill \\ \hfill \mathrm{d}\tilde{Y}& ={|\det (A)|}^{2n}{|\det (B)|}^{2m}\mathrm{d}\tilde{X}=|\det (A{A}^{*}){|}^n{|\det (B^{*}B)|}^m\mathrm{d}\tilde{X}\hfill \end{array}$$

where $|\det (·)|$ denotes the absolute value of the determinant of $(·)$.

The proof of Lemma 1 and other lemmas to follow may be seen from [11]. When a $m\times m$ matrix X is symmetric, $X=X^{\prime }$, then we have a companion result to Lemma 1, which will be stated next.

Lemma 2.

Let $X=X^{\prime }$ be a symmetric $m\times m$ matrix, and let A be a $m\times m$ constant nonsingular matrix. Then, we have the following:

$$Y=AX{A}^{\prime },|A|\ne 0,\Rightarrow \mathrm{d}Y={|A|}^{p+1}\mathrm{d}X$$

and when a $m\times m$ matrix $\tilde{X}={\tilde{X}}^{*}$ in the complex domain is Hermitian, as well as when A is a $m\times m$ nonsingular constant matrix in the real or complex domain, then we have the following:

$$\tilde{Y}=A\tilde{X}{A}^{*},|A|\ne 0,\Rightarrow \mathrm{d}\tilde{Y}={|\det (A)|}^{2m}\mathrm{d}\tilde{X}={|\det (A{A}^{*})|}^m\mathrm{d}\tilde{X}.$$

Now, under Lemma 1, (1) reduces to an integral over Y. Let us denote it as $f_1(Y)$. Then, we have the following:

$${\int }_Y{f}_1(Y)\mathrm{d}Y={c|A|}^{-\frac{q}{2}}{|B|}^{-\frac{p}{2}}{\int }_Y{|Y{Y}^{\prime }|}^{\gamma }{[\mathrm{tr}(Y{Y}^{\prime })]}^{\eta }{\mathrm{e}}^{-\alpha {[\mathrm{tr}(Y{Y}^{\prime })]}^{\delta }}\mathrm{d}Y.$$

(3)

The corresponding integral in the complex case is the following:

$${\int }_{\tilde{Y}}{f}_{1c}(\tilde{Y})\mathrm{d}\tilde{Y}=\tilde{c}|\det (A){|}^{-q}|\det (B){|}^{-p}{\int }_{\tilde{Y}}{|\det (\tilde{Y}{\tilde{Y}}^{*})|}^{\gamma }{[\mathrm{tr}(\tilde{Y}{\tilde{Y}}^{*})]}^{\eta }{\mathrm{e}}^{-\alpha {[\mathrm{tr}(\tilde{Y}{\tilde{Y}}^{*})]}^{\delta }}\mathrm{d}\tilde{Y}.$$

(4)

The function $f_1(Y)$ in the real domain, when $\gamma =0,\eta \ne 0,$ and $\delta \ne 1$, is often referred to as a Kotz model by most of the authors who use such a model. When the exponent of the determinant is $\gamma \ne 0$, then the evaluation of the integral over $f_1(Y)$ is very difficult, which will be seen from the computations to follow. When $\gamma \ne 0$, and, in the real domain, ref. [6] also calls the model a Kotz model, the normalizing constant given by them and claimed to be available in the earlier literature does, nevertheless, not seem to be correct. The correct normalizing constant and its evaluation in the real and complex domains will be given in detail below. Since $f_2(Y)$ involves a determinant and a trace, where the determinant is a product of the eigenvalues and the trace is a sum, two elementary symmetric functions, if there is a transformation involving elementary symmetric functions, then, one can handle the determinant and trace together. This author does not know of any such transformation. Going through the eigenvalues does not seem to be a good option, because the Jacobian element will involve a Vandermonde determinant and is not very convenient to handle. The next possibility is triangularization, and, in this case, one can also make the determinant be a product of the scalar variables and trace be a sum. Then, one can use a general polar coordinate transformation so that the trace becomes a single variable, namely, the radial variable r, and in the product of determinant and trace, r and sine and cosine product will also as separated. Hence, this approach will as a convenient one. Continuing with the evaluation of (3) in the real case, we have the following situations: if $\delta =1$ and $\eta =0$, or $\gamma =0$, then one would immediately convert $\mathrm{d}X$ into $\mathrm{d}S,S=X{X}^{\prime }$ and integrate it out by using a real matrix-variate gamma integral; in the case of $\eta =0$ and $\delta =1$; one would integrate it out by using the scalar variable gamma integral for $\gamma =0$ and matrix-variate gamma integral for $\gamma >0$. This conversion can be done with the help of Lemma 3, which is given below.

Lemma 3.

Let the $m\times n,m\le n$ matrix X of rank m be in the real domain with $mn$ distinct elements $x_{ij}$. Let the $m\times m$ matrix be denoted by $S=X{X}^{\prime }$, which is positive definite. Then, going through a transformation involving a lower triangular matrix with positive diagonal elements and a semiorthonormal matrix and after integrating out the differential element corresponding to the semiorthonormal matrix, we will have the following connection between $\mathrm{d}X$ and $\mathrm{d}S$; see the following details from [11]:

$$\mathrm{d}X=\frac{{\pi }^{\frac{mn}{2}}}{{\Gamma }_m(\frac{n}{2})}{|S|}^{\frac{n}{2}-\frac{m+1}{2}}\mathrm{d}S$$

where, for example, ${\Gamma }_m(\alpha )$ is the real matrix-variate gamma function given by

$$\begin{array}{cc}\hfill {\Gamma }_m(\alpha )& ={\pi }^{\frac{m(m-1)}{4}}\Gamma (\alpha )\Gamma (\alpha -\frac{1}{2})\dots \Gamma (\alpha -\frac{m-1}{2}),\Re (\alpha )>\frac{m-1}{2}\hfill \\ \hfill & ={\int }_{Z>O}{|Z|}^{\alpha -\frac{m+1}{2}}{\mathrm{e}}^{-\mathrm{tr}(Z)}\mathrm{d}Z,\Re (\alpha )>\frac{m-1}{2}\hfill \end{array}$$

where $\mathrm{tr}(·)$ means the trace of the square matrix $(·)$. Since ${\Gamma }_m(\alpha )$ is associated with the above real matrix-variate gamma integral, we call ${\Gamma }_m(\alpha )$ a real matrix-variate gamma function. This ${\Gamma }_m(\alpha )$ is also known by different names in the literature. When the $m\times n,m\le n$ matrix $\tilde{X}$ of rank m, with distinct elements, is in the complex domain, and we let $\tilde{S}=\tilde{X}{\tilde{X}}^{*}$, which is $m\times m$ and Hermitian positive definite, then, by going through a transformation involving a lower triangular matrix with real and positive diagonal elements and a semiunitary matrix, we can establish the following connection between $\mathrm{d}\tilde{X}$ and $\mathrm{d}\tilde{S}$; please refer to [11]:

$$\mathrm{d}\tilde{X}=\frac{{\pi }^{mn}}{{\tilde{\Gamma }}_m(n)}{|\det (\tilde{S})|}^{n-m}\mathrm{d}\tilde{S}$$

where, for example, ${\tilde{\Gamma }}_m(\alpha )$ is the complex matrix-variate gamma function given by

$$\begin{array}{cc}\hfill {\tilde{\Gamma }}_m(\alpha )& ={\pi }^{\frac{m(m-1)}{2}}\Gamma (\alpha )\Gamma (\alpha -1)\dots \Gamma (\alpha -m+1),\Re (\alpha )>m-1\hfill \\ \hfill & ={\int }_{\tilde{Z}>O}{|\det (\tilde{Z})|}^{\alpha -m}{\mathrm{e}}^{-\mathrm{tr}(\tilde{Z})}\mathrm{d}\tilde{Z},\Re (\alpha )>m-1.\hfill \end{array}$$

We call ${\tilde{\Gamma }}_m(\alpha )$ the complex matrix-variate gamma, because it is associated with the above matrix-variate gamma integral in the complex domain.

However, in our (3), both the determinant and trace enter as multiplicative factors, and there is an exponent $\delta >0$ for the exponential trace. In order to tackle this situation, we will convert $\mathrm{d}X$ to $\mathrm{d}T$, where T is a lower triangular matrix, by using Theorem 2.14 of [11], which is restated here as a lemma. The idea is that, in this case, $|X{X}^{\prime }|=|T{T}^{\prime }|$ becomes product of the squares of the diagonal elements in T only, and $\mathrm{tr}(T{T}^{\prime })$ is also a sum of squares. This conversion can also be achieved by converting the $\mathrm{d}S$ from Lemma 3 to a $\mathrm{d}T$ by using another result, where T is lower triangular.

Lemma 4.

Let X be a $m\times n,m\le n$ matrix of rank m with functionally independent $mn$ real scalar variables as elements. Let T be a lower triangular matrix, and let $U_1$ be a semiorthonormal matrix $U_1{U}_1^{\prime }=I_m$. Consider the transformation $X=T{U}_1$, where both T and $U_1$ are uniquely selected, for example, with the diagonal elements positive in T and with the first column elements that are positive in $U_1$. Then, after integrating out the differential element corresponding the semiorthonormal matrix $U_1$, one has the following connection between $\mathrm{d}X$ and $\mathrm{d}T$; please refer to [11]:

$$X=T{U}_1\Rightarrow \mathrm{d}X=\frac{{\pi }^{\frac{mn}{2}}}{{\Gamma }_m(\frac{n}{2})}\{\prod _{j=1}^m|t_{jj}{|}^{n-j}\}\mathrm{d}T$$

and in the complex case, let $\tilde{X}$ be a $m\times n,m\le n$ matrix of rank m with $mn$ distinct elements in the complex domain. Let $\tilde{T}$ be a lower triangular matrix in the complex domain with the diagonal elements that are real and positive, and let ${\tilde{U}}_1$ be a semiunitary matrix, ${\tilde{U}}_1{\tilde{U}}_1^{*}=I_m$, where $\tilde{T}$ and ${\tilde{U}}_1$ are uniquely chosen. Then, after integrating out the differential element corresponding to ${\tilde{U}}_1$, one has the following connection between $\mathrm{d}\tilde{X}$ and $\mathrm{d}\tilde{T}$:

$$\tilde{X}=\tilde{T}{\tilde{U}}_1\Rightarrow \mathrm{d}\tilde{X}=\frac{{\pi }^{mn}}{{\tilde{\Gamma }}_m(n)}\{\prod _{j=1}^m{(t_{jj})}^{2(n-j)+1}\}\mathrm{d}\tilde{T}.$$

Let us consider the evaluation of (3) in the real case first. By converting the $\mathrm{d}Y$ in (3) to a $\mathrm{d}T$ by using Lemma 4, the integral part of (3) over Y is the following, which is denoted by $f_2(T)$:

$$\begin{array}{c}{\int }_T{f}_2(T)\mathrm{d}T={c|A|}^{-\frac{q}{2}}{|B|}^{-\frac{p}{2}}\frac{{\pi }^{\frac{pq}{2}}}{{\Gamma }_p(\frac{q}{2})}{\int }_T{|T{T}^{\prime }|}^{\gamma }{[\mathrm{tr}(T{T}^{\prime })]}^{\eta }\\ \times {\mathrm{e}}^{-\alpha {[\mathrm{tr}(T{T}^{\prime })]}^{\delta }}\{{\displaystyle \prod _{j=1}^p}|t_{jj}{|}^{q-j}\}\mathrm{d}T.\end{array}$$

(5)

The corresponding equation in the complex domain is the following:

$$\begin{array}{c}{\int }_{\tilde{T}}{f}_{2c}(\tilde{T})\mathrm{d}\tilde{T}=\tilde{c}|\det (A){|}^{-q}|\det (B){|}^{-p}\frac{{\pi }^{pq}}{{\Gamma }_p(q)}{\int }_{\tilde{T}}{|\det (\tilde{T}{\tilde{T}}^{*})|}^{\gamma }{[\mathrm{tr}(\tilde{T}{\tilde{T}}^{*})]}^{\eta }\\ \times {\mathrm{e}}^{-\alpha {[\mathrm{tr}(\tilde{T}{\tilde{T}}^{*})]}^{\delta }}\{{\displaystyle \prod _{j=1}^p}{t}_{jj}^{2(q-j)+1}\}\mathrm{d}\tilde{T}.\end{array}$$

(6)

Note that in the real case, we have the following:

$$|T{T}^{\prime }|=\prod _{j=1}^p{t}_{jj}^2,\mathrm{tr}(T{T}^{\prime })=\sum _{j=1}^p{t}_{jj}^2+\sum _{i>j}{t}_{ij}^2$$

where in ${\sum }_{j=1}^p{t}_{jj}^2$, there are p terms, and the second sum has $p(p-1)/2$ terms; thus, we have a total of $k=p(p+1)/2$ terms. The corresponding quantity in the complex domain is the following:

$$|\det (\tilde{T}{\tilde{T}}^{*})|=\prod _{j=1}^p{t}_{jj}^2,\mathrm{tr}(\tilde{T}{\tilde{T}}^{*})=\sum _{j=1}^p{t}_{jj}^2+\sum _{j>k}|{\tilde{t}}_{jk}{|}^2,{|{\tilde{t}}_{jk}|}^2=t_{jk1}^2+t_{jk2}^2$$

where ${\tilde{t}}_{jk}=t_{jk1}+i{t}_{jk2},i=\sqrt{(-1)},t_{jk1},$ and $t_{jk2}$ are real, and, in the first sum, there are p square terms, but, in the sum ${\sum }_{j>k}{|{\tilde{t}}_{jk}|}^2$, there are a total of $2[\frac{p(p-1)}{2}]=p(p-1)$ square terms; thus, there are a total of $p^2$ square terms in the complex case.

Let us consider a polar coordinate transformation in the real case on all of the $k=p(p+1)/2$ terms by using the transformation on page 44 of [12], which is restated here for convenience, that is, $\{t_{11},t_{22},\dots ,t_{pp},t_{21},\dots ,t_{pp-1}\}\to \{r,{\theta }_1,\dots ,{\theta }_{k-1}\}$, and $k=p(p+1)/2$.

$$\begin{array}{cc}\hfill t_{11}& =rsin{\theta }_1\hfill \\ \hfill t_{22}& =rcos{\theta }_{1}sin{\theta }_2\hfill \\ \hfill ⋮\\ \hfill t_{pp}& =rcos{\theta }_1\dots cos{\theta }_{p-1}sin{\theta }_p\hfill \\ \hfill t_{21}& =rcos{\theta }_1\dots cos{\theta }_{p}sin{\theta }_{p+1}\hfill \\ \hfill ⋮\\ \hfill t_{pp-1}& =rcos{\theta }_1\dots cos{\theta }_{k-1}\hfill \end{array}$$

(7)

for $-\frac{\pi }{2}<{\theta }_j\le \frac{\pi }{2},j=1,\dots ,k-2;-\pi <{\theta }_{k-1}\le \pi$ for $k=p(p+1)/2$ in the real case, and $k=p^2$ in the complex case. The structure of the polar coordinate transformation in the complex case remains as in the real case. The only change is that in the real case $k=p(p+1)/2$ and in the complex case $k=p^2$. The Jacobian of the transformation in the real case is the following:

$$\mathrm{d}{t}_{11}\wedge \dots \wedge \mathrm{d}{t}_{pp-1}=r^{k-1}\{\prod _{j=1}^{k-1}|cos{\theta }_j{|}^{k-1-j}\}\mathrm{d}r\wedge \mathrm{d}{\theta }_1\wedge \dots \wedge \mathrm{d}{\theta }_{k-1},k=p(p+1)/2$$

and, in the complex case, the Jacobian is given by the following:

$$\mathrm{d}{t}_{11}\wedge \dots \wedge \mathrm{d}{t}_{pp}\wedge \dots \wedge \mathrm{d}{t}_{pp-12}=r^{p^2-1}\{\prod _{j=1}^{p^2-1}|cos{\theta }_j{|}^{p^2-j-1}\}\mathrm{d}r\wedge \mathrm{d}{\theta }_1\wedge \dots \wedge \mathrm{d}{\theta }_{p^2-1}$$

for the same ranges for ${\theta }_j$ as in the real case, but, in the complex case, $k=p^2$.

The normalizing constant c in the real case coming from (5) is quoted in [6] by citing earlier works. However, none of them seems to have given an evaluation of the integral in (5) explicitly. The normalizing constant c given in [6] does not seem to be correct. Since the integral in (5) appears in very many places as a Kotz integral and is used in many disciplines, a detailed evaluation of the integral in (5) is warranted. In addition, no work seems to have given $\tilde{c}$ in the complex case. Hence, the evaluations of c and $\tilde{c}$ in the real and complex cases, respectively, will be given here in detail.

Evaluation of the Integral in (5) in the Real Case and (6) in the Complex Case

Note that ${\sum }_{ij}{t}_{ij}^2=r^2$. From the Jacobian part, the factor containing r is $r^{k-1}={(r^2)}^{\frac{k}{2}-\frac{1}{2}}$. In the product ${\prod }_{j=1}^p{t}_{jj}^2$, each $t_{jj}^2$ contains an $r^2$. In addition, the Jacobian part is ${\prod }_{j=1}^p|t_{jj}{|}^{q-j}={\prod }_{j=1}^p{(t_{jj}^2)}^{\frac{q}{2}-\frac{j}{2}}={|T{T}^{\prime }|}^{\frac{q}{2}-\frac{j}{2}}$. Upon collecting all r values, the exponent of $r^2$ in the real case is the following:

$$(\gamma +\frac{q}{2}-\frac{1}{2})+(\gamma +\frac{q}{2}-\frac{2}{2})+\dots +(\gamma +\frac{q}{2}-\frac{p}{2})+\eta +\frac{p(p+1)}{4}-\frac{1}{2}=p(\gamma +\frac{q}{2})+\eta -\frac{1}{2}.$$

Then, integration over r gives the following:

$${\int }_0^{\infty }{(r^2)}^{p((\gamma +\frac{q}{2})+\eta )-\frac{1}{2}}{\mathrm{e}}^{-\alpha {(r^2)}^{\delta }}\mathrm{d}r=\frac{1}{2\delta }\Gamma [\frac{1}{\delta }(p(\gamma +\frac{q}{2})+\eta )]{\alpha }^{-[\frac{1}{\delta }(p(\gamma +\frac{q}{2})+\eta )]}$$

(8)

for $\Re (\gamma )>-\frac{q}{2},\Re (\eta )>0,\alpha >0,$ and $\delta >0$. The corresponding integral over r in the complex domain is the following:

$${\int }_0^{\infty }{(r^2)}^{p(\gamma +q)+\eta -\frac{1}{2}}{\mathrm{e}}^{-\alpha {(r^2)}^{\delta }}\mathrm{d}r=\frac{1}{2\delta }\Gamma [\frac{1}{\delta }(p(\gamma +q)+\eta )]{\alpha }^{-\frac{1}{\delta }(p(\gamma +q)+\eta )}$$

(9)

for $\Re (\gamma )>-q,\delta >0,\alpha >0,$ and $\Re (\eta )>0$.

2.3. Evaluation of the Sine and Cosine Product in the Real Case

Consider the integration of the factors containing the ${\theta }_j$s in the real case. These ${\theta }_j$s come from ${\prod }_{j=1}^p{t}_{jj}^2$ and from the Jacobian part. Consider ${\theta }_1$. The exponent of ${sin}^2{\theta }_1$ is $\gamma +\frac{q}{2}-\frac{1}{2}$. The exponent of ${cos}^2{\theta }_1$ is $(\gamma +\frac{q}{2}-\frac{2}{2})+(\gamma +\frac{q}{2}-\frac{3}{2})+\dots +(\gamma +\frac{q}{2}-\frac{p}{2})=(p-1)(\gamma +\frac{q}{2})-\frac{p(p+1)}{4}+\frac{1}{2}$, and the part coming from the Jacobian is $|cos{\theta }_1{|}^{k-1-1}$. Note that $|cos{\theta }_1{|}^{k-1-1}={({cos}^2{\theta }_1)}^{\frac{p(p+1)}{4}-\frac{3}{2}}|cos{\theta }_1|$. Then, the integral over over ${\theta }_1$, denoting the integral over ${\theta }_1$ as $I_{{\theta }_1}$, gives the following, where in all the integrations over the ${\theta }_j$s to follow, we will use the transformations $x=sin{\theta }_j$ and $u=x^2$:

$$\begin{array}{cc}\hfill I_{{\theta }_1}& ={\int }_{-\frac{\pi }{2}}^{\frac{\pi }{2}}{({sin}^2{\theta }_1)}^{\gamma +\frac{q}{2}-\frac{1}{2}}{({cos}^2{\theta }_1)}^{(p-1)(\gamma +\frac{q}{2})-1}|cos{\theta }_1|\mathrm{d}{\theta }_1\hfill \\ \hfill & =2{\int }_0^{\frac{\pi }{2}}{({sin}^2{\theta }_1)}^{\gamma +\frac{q}{2}-\frac{1}{2}}{({cos}^2{\theta }_1)}^{(p-1)(\gamma +\frac{q}{2})-1}|cos{\theta }_1|\mathrm{d}{\theta }_1\hfill \\ \hfill & =2{\int }_0^1{(x^2)}^{\gamma +\frac{q}{2}-\frac{1}{2}}{(1-x^2)}^{(p-1)(\gamma +\frac{q}{2})-1}\mathrm{d}x={\int }_0^1{u}^{\gamma +\frac{q}{2}-1}{(1-u)}^{(p-1)(\gamma +\frac{q}{2})-1}\mathrm{d}u\hfill \\ \hfill & =\frac{\Gamma (\gamma +\frac{q}{2})\Gamma [(p-1)(\gamma +\frac{q}{2})]}{\Gamma [p(\gamma +\frac{q}{2})]},\Re (\gamma )>-\frac{q}{2}.\hfill \end{array}$$

(10)

Now, by collecting all of the factors containing ${\theta }_2$ and proceeding as in the case of ${\theta }_1$, we have the following result for the integral over ${\theta }_2$:

$$I_{{\theta }_2}=\frac{\Gamma (\gamma +\frac{q}{2}-\frac{1}{2})\Gamma [(p-2)(\gamma +\frac{q}{2})+\frac{1}{2}]}{\Gamma [(p-1)(\gamma +\frac{q}{2})]},\Re (\gamma )>-\frac{q}{2}+\frac{1}{2}.$$

(11)

Note that the denominator gamma in (11) cancels with one numerator gamma in (10). The pattern of cancellation of the denominator gamma in the next step with one numerator gamma in its previous step will continue leaving only one factor in the numerator and no factor in the denominator, except the very first step involving (10) and (11), where the first denominator gamma, namely, $\Gamma (p(\gamma +\frac{q}{2}))$, is left out. When integrating ${\theta }_{p-1}$, we have the following:

$$I_{{\theta }_{p-1}}=\frac{\Gamma (\gamma +\frac{q}{2}-\frac{p-2}{2})\Gamma ((\gamma +\frac{q}{2})-\frac{p}{2}-\frac{p-1}{2}+\frac{p(p+1)}{4})}{\Gamma (2(\gamma +\frac{q}{2})-\frac{p-2}{2}-\frac{p-1}{2}-\frac{p}{2}+\frac{p(p+1)}{4})},\Re (\gamma )>-\frac{q}{2}+\frac{p-2}{2}.$$

(12)

Note that, when considering ${\theta }_p$, there is no cosine factor coming from $t_{pp}$, and the cosine factor comes only from the Jacobian part. We can see that

$$I_{{\theta }_p}=\frac{\Gamma (\gamma +\frac{q}{2}-\frac{p-1}{2})\Gamma (\frac{p(p+1)}{4}-\frac{p}{2})}{\Gamma ((\gamma +\frac{q}{2})-\frac{p}{2}-\frac{p-1}{2}+\frac{p(p+1)}{4})},\Re (\gamma )>-\frac{q}{2}+\frac{p-1}{2}.$$

(13)

Again, the denominator gamma in (13) cancels with one numerator gamma in (12). This pattern will continue for $k=p+1,p+2,\dots$. For the integrals over ${\theta }_j,j=p+1,p+2,\dots$ the only contribution is from the Jacobian part; no sine factor will be there. Consider ${\theta }_{p+1}$. We see that

$$I_{{\theta }_{p+1}}=\frac{\Gamma (\frac{1}{2})\Gamma (\frac{p(p+1)}{4}-\frac{p+1}{2})}{\Gamma (\frac{p(p+1)}{4}-\frac{p}{2})},p>2.$$

(14)

Again, cancellation will hold. Now, consider a few last cases of ${\theta }_j$. For $j=\frac{p(p+1)}{2}-3=k-3$, we have

$$I_{{\theta }_{k-3}}=\frac{\Gamma (\frac{1}{2})\Gamma (\frac{3}{2})}{\Gamma (2)}$$

and for $j=k-2$, we have

$$I_{{\theta }_{k-2}}=\frac{\Gamma (\frac{1}{2})\Gamma (1)}{\Gamma (\frac{3}{2})}$$

and the last ${\theta }_j$ goes from $-\pi$ to $\pi$ with no contribution from the Jacobian part; hence, we have the following:

$$I_{{\theta }_{k-1}}=2\pi .$$

Note that, starting from $j=p+1$ to $j=k-2$, the gamma factor left in the numerator is $\Gamma (\frac{1}{2})=\sqrt{\pi }$. There are $\frac{p(p-1}{2}-2$ such factors, and the last one is $2\pi$; thus, the product is $2{\pi }^{\frac{p(p-1)}{4}}$. For $j=1,\dots ,p$, the factors left out in the numerator are $\Gamma (\gamma +\frac{q}{2})\Gamma (\gamma +\frac{q}{2}-\frac{1}{2})\dots \Gamma (\gamma +\frac{q}{2}-\frac{p-1}{2})$, and for $j=p+1,\dots ,k-1$, we have ${\pi }^{\frac{p(p-1)}{4}}$, thus giving ${\Gamma }_p(\gamma +\frac{q}{2}),\Re (\gamma )>-\frac{q}{2}+\frac{p-1}{2}$. For $j=1$, there is one gamma left in the denominator, namely, $\Gamma (p(\gamma +\frac{q}{2}))$. Taking the product of the integral over all the ${\theta }_j$s in the real case yields the following:

$$2{\Gamma }_p(\gamma +\frac{q}{2})/\Gamma (p(\gamma +\frac{q}{2}))$$

(15)

where ${\Gamma }_p(·)$ is the real matrix-variate gamma defined in Lemma 3.

Evaluation of the Integral over the ${\theta }_j$s in the Complex Case

The sine and cosine functions come from the transformations corresponding to $t_{11},\dots ,t_{pp}$ from the Jacobian when going from $\tilde{X}$ to $\tilde{T}$ and from the Jacobian in the polar coordinate transformation. The Jacobian part in the polar coordinate transformation is the following:

$$\begin{array}{c}{\displaystyle \prod _{j=1}^{p^2-1}}|cos{\theta }_j{|}^{p^2-j-1}=\{{\displaystyle \prod _{j=1}^{p^2-1}}{({cos}^2{\theta }_j)}^{\frac{p^2}{2}-\frac{j}{2}-1}|cos{\theta }_j|\}\\ |\tilde{T}{\tilde{T}}^{*}{|}^{\gamma }[{\displaystyle \prod _{j=1}^p}{t}_{jj}^{2(q-j)+1}]={\displaystyle \prod _{j=1}^p}{(t_{jj}^2)}^{\gamma +q-j+\frac{1}{2}}.\end{array}$$

(16)

By collecting factors containing ${\theta }_1$, we observe that $sin{\theta }_1$ comes from $t_{11}$, and $cos{\theta }_1$ comes from $t_{22},\dots ,t_{pp}$ and the Jacobian part. The exponent of ${sin}^2{\theta }_1$ is $\gamma +q-\frac{1}{2}$, and the exponent of ${cos}^2{\theta }_1$ is $(p-1)(\gamma +q+\frac{1}{2})-\frac{p(p+1)}{2}+1+\frac{p^2}{2}-\frac{1}{2}-1=(p-1)(\gamma +q)-1$. In all of the integrals to follow, ${\int }_{-\frac{\pi }{2}}^{\frac{\pi }{2}}(·)\mathrm{d}\theta =2{\int }_0^{\frac{\pi }{2}}(·)\mathrm{d}\theta$ due to the evenness of the integrand. Then, we will use the transformations $x=sin\theta$ and $u=x^2$; the steps are parallel to those in the real case. Therefore, we have the following:

$$\begin{array}{c}{\int }_{-\frac{\pi }{2}}^{\frac{\pi }{2}}{({sin}^2{\theta }_1)}^{\gamma +q-\frac{1}{2}}{({cos}^2{\theta }_1)}^{(p-1)(\gamma +q)-1}|cos{\theta }_1|\mathrm{d}{\theta }_1=\frac{\Gamma (\gamma +q)\Gamma ((p-1)(\gamma +q))}{\Gamma (p(\gamma +q))},\end{array}$$

(17)

for $\Re (\gamma )>-q$. By collecting the factors containing ${\theta }_2$, we note that the exponent of ${sin}^2{\theta }_2$ is $\gamma +q-2+\frac{1}{2}$, and the exponent of ${cos}^2{\theta }_2$ is $(p-2)(\gamma +q+\frac{1}{2})-\frac{p(p+1)}{2}+1+2+\frac{p^2}{2}-2=(p-2)(\gamma +q)$. Hence, we have the following:

$$\begin{array}{c}2{\int }_0^{\frac{\pi }{2}}{({sin}^2{\theta }_2)}^{(\gamma +q)-\frac{3}{2}}{({cos}^2{\theta }_2)}^{(p-2)(\gamma +q)+1-1}|cos{\theta }_2|\mathrm{d}{\theta }_2\\ =\frac{\Gamma (\gamma +q-1)\Gamma ((p-2)(\gamma +q)+1)}{\Gamma ((p-1)(\gamma +q))},\Re (\gamma )>-q+1.\end{array}$$

(18)

Note that $\Gamma ((p-1)(\gamma +q))$ from the denominator of (18) cancels with the same in the numerator of (17) to leave one gamma, namely, $\Gamma (\gamma +q)$ in the numerator of (17) and one gamma, namely, $\Gamma (p(\gamma +q))$ in the denominator of (17). The pattern of cancellation of the gamma in the denominator of a step canceling with a gamma in the numerator of the previous step will continue as seen in the real case. Let us check for $j=p$ and $j=p+1$ to see whether the pattern is continuing, where in $j=p+1$, there is no contribution of the sine function, and the only cosine function is coming from the Jacobian part. For $j=p$, we have the following:

$$\begin{array}{c}{\int }_{{\theta }_p}{({sin}^2{\theta }_p)}^{\gamma +q+\frac{1}{2}-p}{({cos}^2{\theta }_p)}^{\frac{p(p-1)}{2}-1}|cos{\theta }_p|\mathrm{d}{\theta }_p=\frac{\Gamma (\gamma +q-p+1)\Gamma (\frac{p(p-1)}{2})}{\Gamma (\gamma +1-p+1+\frac{p(p-1)}{1})}.\end{array}$$

(19)

For $j=p+1$, we have the following:

$$\begin{array}{c}{\int }_{{\theta }_{p+1}}{({cos}^2{\theta }_{p+1})}^{\frac{p^2-p-3}{2}}|cos{\theta }_{p+1}|\mathrm{d}{\theta }_{p+1}=\frac{\Gamma (\frac{1}{2})\Gamma (\frac{p(p-1)}{2}-\frac{1}{2})}{\Gamma (\frac{p(p-1)}{2})}.\end{array}$$

(20)

The pattern of cancelation is continuing. However, starting from $j=p+1,\dots ,p^2-2$, the factor left out in the numerator is $\Gamma (\frac{1}{2})=\sqrt{\pi }$, and the last factor gives $2\pi$, because the range here is $-\pi <{\theta }_{p^2-1}\le \pi$, and, hence, the factors left out in the numerator are ${(\sqrt{\pi })}^{p(p-1)}\Gamma (\gamma +q)\Gamma (\gamma +q-1)\dots \Gamma (\gamma +q-p+1)={\tilde{\Gamma }}_p(\gamma +q),\Re (\gamma )>-q+p-1$, and one gamma, namely, $\Gamma (p(\gamma +q))$ is left out in the denominator of the integration over ${\theta }_1$. Hence, the integration over all of the sine and cosine functions in the complex case is the following:

$$\begin{array}{c}\frac{2{\tilde{\Gamma }}_p(\gamma +q)}{\Gamma (p(\gamma +q))}\end{array}$$

(21)

where ${\tilde{\Gamma }}_p(·)$ is the complex matrix-variate gamma function defined in Lemma 3. Then, the final result of integration over r and the integration over all of the ${\theta }_j$s in the real case is the following:

$$\begin{array}{cc}\hfill {\int }_{X}f(X)\mathrm{d}X& ={\int }_{X}c{|A^{\frac{1}{2}}(X-M)B{(X-M)}^{\prime }{A}^{\frac{1}{2}}|}^{\gamma }{[\mathrm{tr}(A^{\frac{1}{2}}(X-M)B{(X-M)}^{\prime }{A}^{\frac{1}{2}})]}^{\eta }\hfill \\ \hfill & \times {\mathrm{e}}^{-\alpha {[\mathrm{tr}(A^{\frac{1}{2}}(X-M)B{(X-M)}^{\prime }{A}^{\frac{1}{2}})]}^{\delta }}\mathrm{d}X\hfill \\ \hfill & =c\frac{1}{2\delta }\Gamma [\frac{1}{\delta }(p(\gamma +\frac{q}{2})+\eta )]{\alpha }^{-\frac{1}{\delta }[p(\gamma +\frac{q}{2})+\eta ]}\hfill \\ \hfill & \times 2\frac{{\Gamma }_p(\gamma +\frac{q}{2})}{\Gamma (p(\gamma +\frac{q}{2}))}{|A|}^{-\frac{q}{2}}{|B|}^{-\frac{p}{2}}\frac{{\pi }^{\frac{pq}{2}}}{{\Gamma }_p(\frac{q}{2})}\hfill \\ \hfill & =c\frac{{|A|}^{-\frac{q}{2}}{|B|}^{-\frac{p}{2}}{\pi }^{\frac{pq}{2}}}{\delta {\Gamma }_p(\frac{q}{2}){\alpha }^{\frac{1}{\delta }(p(\gamma +\frac{q}{2})+\eta )}}\frac{\Gamma [\frac{1}{\delta }(p(\gamma +\frac{q}{2})+\eta )]{\Gamma }_p(\gamma +\frac{q}{2})}{\Gamma (p(\gamma +\frac{q}{2}))}\hfill \end{array}$$

(22)

where $\Re (\gamma )>-\frac{q}{2}+\frac{p-1}{2},\Re (\eta )>0,\delta >0,\alpha >0,p\le q,M=E[X]$, $A>O,$ and $B>O$ are constant matrices, where A is a $p\times p$, B is a $q\times q$, and X is a $p\times q,p\le q$ real matrix of rank p; the corresponding integral in the complex case is given by the following:

$$\begin{array}{c}{\int }_{\tilde{X}}{f}_c(\tilde{X})\mathrm{d}\tilde{X}=\tilde{c}\frac{{\pi }^{pq}}{\delta {\tilde{\Gamma }}_p{(q)|\det (A)|}^q{|\det (B)|}^p{\alpha }^{\frac{1}{\delta }(p(\gamma +q)+\eta )}}\frac{\Gamma [\frac{1}{\delta }(p(\gamma +q)+\eta )]{\tilde{\Gamma }}_p(\gamma +q)}{\Gamma (p(\gamma +q))}\end{array}$$

(23)

for $\Re (\gamma )>-q+p-1,\Re (\eta )>0,\delta >0,\alpha >0,\tilde{M}=E[\tilde{X}],A=A^{*}>O,B=B^{*}>O,$ and $p\le q$. Therefore, the normalizing constants c and $\tilde{c}$ are the following:

$$\begin{array}{c}c=\frac{{|A|}^{\frac{q}{2}}{|B|}^{\frac{p}{2}}\delta \Gamma (p(\gamma +\frac{q}{2})){\alpha }^{\frac{1}{\delta }(p(\gamma +\frac{q}{2})+\eta )}{\Gamma }_p(\frac{q}{2})}{{\Gamma }_p(\gamma +\frac{q}{2})\Gamma [\frac{1}{\delta }(p(\gamma +\frac{q}{2})+\eta )]{\pi }^{\frac{pq}{2}}},\Re (\gamma )>-\frac{q}{2}+\frac{p-1}{2},\end{array}$$

(24)

for $\delta >0,\alpha >0,\Re (\eta )>0,$ and $p\le q$; likewise, we have the following:

$$\begin{array}{c}\tilde{c}=\frac{{|\det (A)|}^q{|\det (B)|}^p\delta {\tilde{\Gamma }}_p(q){\alpha }^{\frac{1}{\delta }(p(\gamma +q)+\eta )}\Gamma (p(\gamma +q))}{{\pi }^{pq}{\tilde{\Gamma }}_p(\gamma +q)\Gamma [\frac{1}{\delta }(p(\gamma +q)+\eta )]},\Re (\gamma )>-q+p-1\end{array}$$

(25)

for $\delta >0,\alpha >0,\Re (\eta )>0,$ and $p\le q$.

From the general results in (22) and (23) we can have the following interesting special cases: $A=I,\eta =0$; $\eta =0,B=I$; $A=I,B=I,\eta =0$; $\gamma =0$; $\gamma =0,\eta =0$; and $\eta =0,\delta =1$. Since the integral over the sine and cosine product is very important in many types of applications, we will give these as theorems here.

Theorem 1.

Let ${\Theta }_k=\{{\theta }_1,\dots ,{\theta }_{k-1}\},$ and let $k=\frac{p(p+1)}{2}$ in the real case; the integral over ${\Theta }_k$, denoted by $I_{{\Theta }_k}$, is the following:

$$\begin{array}{cc}\hfill I_{{\Theta }_k}& ={\int }_{\Theta }\{\prod _{j=1}^p{(co{s}^2{\theta }_1{cos}^2{\theta }_2\dots {cos}^2{\theta }_{j-1}{sin}^2{\theta }_j)}^{\gamma +\frac{q}{2}-\frac{j}{2}}|cos{\theta }_j{|}^{k-1-j}\}\{\prod _{j=p+1}^{k-1}|cos{\theta }_j{|}^{k-1-j}\}\mathrm{d}\Theta \hfill \\ \hfill & =\frac{2{\Gamma }_p(\gamma +\frac{q}{2})}{\Gamma (p(\gamma +\frac{q}{2}))},\Re (\gamma )>-\frac{q}{2}+\frac{p-1}{2},p\le q,p\ge 2,k=\frac{p(p+1)}{2}.\hfill \end{array}$$

The corresponding result in the complex case is the following, where ${\Theta }_k$ here has the same format as in the real case, but here $k=p^2$.

Theorem 2.

Let ${\Theta }_k=\{{\theta }_1,\dots ,{\theta }_{k-1}\},$ and let $k=p^2$. The integral over ${\Theta }_k$ in the complex case is the following:

$$\begin{array}{cc}\hfill I_{{\Theta }_k}& ={\int }_{{\Theta }_k}\{\prod _{j=1}^p{(co{s}^2{\theta }_1{cos}^2{\theta }_1\dots {cos}^2{\theta }_{j-1}{sin}^2{\theta }_j)}^{\gamma +q-j+\frac{1}{2}}{({cos}^2{\theta }_j)}^{\frac{p^2-j-2}{2}}|cos{\theta }_j|\}\hfill \\ \hfill & \times \{\prod _{j=p+1}^{p^2-1}{({cos}^2{\theta }_j)}^{\frac{p^2-j-2}{2}}|cos{\theta }_j|\}\mathrm{d}{\Theta }_k\hfill \\ \hfill & =\frac{2{\tilde{\Gamma }}_p(\gamma +q)}{\Gamma (p(\gamma +q))},\Re (\gamma )>-q+p-1,p\le q,p\ge 2,k=p^2.\hfill \end{array}$$

From (22) in the real case, we have the following theorems:

Theorem 3.

Let Y be a $p\times q,p\le q$ matrix of rank p with the $pq$ elements being functionally independent real scalar variables. For $\delta >0,\Re (\eta )>0,$ and $\Re (\gamma )>-\frac{q}{2}+\frac{p-1}{2}$, we have the following:

$$\begin{array}{cc}\hfill & {\int }_Y{|Y{Y}^{\prime }|}^{\gamma }{[\mathrm{tr}(Y{Y}^{\prime })]}^{\eta }{\mathrm{e}}^{-\alpha {[\mathrm{tr}(Y{Y}^{\prime })]}^{\delta }}\mathrm{d}Y\hfill \\ \hfill & =\frac{{\pi }^{\frac{pq}{2}}}{{\Gamma }_p(\frac{q}{2})}\frac{\Gamma [\frac{1}{\delta }(p(\gamma +\frac{q}{2})+\eta )]{\Gamma }_p(\gamma +\frac{q}{2})}{\delta \Gamma (p(\gamma +\frac{q}{2})){\alpha }^{\frac{1}{\delta }(p(\gamma +\frac{q}{2})+\eta )}}.\hfill \end{array}$$

Remark 1.

In the widely used normalizing constant in [6], which was quoted from earlier references, its corresponding to the normalizing constant in Theorem 3 above does not seem to be correct. The correct one is given in Theorem 3. There are several normalizing constants reported in [6] for various particular cases of Theorem 3. Unfortunately, all of the normalizing constants quoted there, except one, seem to be incorrect. The normalizing constants in [6] as they appear, the corresponding translation in terms of the parameters of the present paper, and the corresponding correct ones are listed in Appendix B at the end of this paper.

Theorem 4.

Let $\tilde{Y}$ be a $p\times q,p\le q$ matrix in the complex domain with rank p, where the $pq$ elements are functionally independent complex scalar variables. For $\delta >0,\alpha >0,\Re (\eta )>0,$ and $\Re (\gamma )>-q+p-1$, we have the following:

$$\begin{array}{cc}\hfill & {\int }_{\tilde{Y}}{|\det (\tilde{Y}{\tilde{Y}}^{*})|}^{\gamma }{[\mathrm{tr}(\tilde{Y}{\tilde{Y}}^{*})]}^{\eta }{\mathrm{e}}^{-\alpha {[\mathrm{tr}(\tilde{Y}{\tilde{Y}}^{*})]}^{\delta }}\mathrm{d}\tilde{Y}\hfill \\ \hfill & =\frac{{\pi }^{pq}}{{\tilde{\Gamma }}_p(q)}\frac{\Gamma (\frac{1}{\delta }(p(\gamma +q)+\eta )){\tilde{\Gamma }}_p(\gamma +q)}{\delta \Gamma (p(\gamma +q)){\alpha }^{\frac{1}{\delta }(p(\gamma +q)+\eta )}}.\hfill \end{array}$$

The details of the derivations are already given in Theorem 2 and in earlier parts. First, by using the Lemma 4 complex part and then using the general polar coordinate transformation for the $p^2$ variables, the variables concerned are transformed in to r and ${\theta }_j$. Then, the r for the complex case is integrated out, and the ${\theta }_j$s are then integrated out by using Theorem 2.

Corollary 1.

This is the corollary to Theorem 3 for $\eta =0$. For $Y,\delta ,\gamma ,$ and α, as defined in Theorem 3, we have the following:

$${\int }_Y{|Y{Y}^{\prime }|}^{\gamma }{\mathrm{e}}^{-\alpha {[\mathrm{tr}(Y{Y}^{\prime })]}^{\delta }}\mathrm{d}Y=\frac{{\pi }^{\frac{pq}{2}}{\Gamma }_p(\gamma +\frac{q}{2})\Gamma (\frac{p}{\delta }(\gamma +\frac{q}{2}))}{\delta {\Gamma }_p(\frac{q}{2})\Gamma (p(\gamma +\frac{q}{2})){\alpha }^{\frac{p}{\delta }(\gamma +\frac{q}{2})}}$$

for $\Re (\gamma )>-\frac{q}{2}+\frac{p-1}{2},\delta >0,$ and $p\le q$.

The results quoted from some of the earlier works of others and reported in [6], corresponding to our Theorem 3 and Corollary 1, do not agree with our results; see Appendix B at the end of this paper.

The result corresponding to Corollary 1 in the complex case is the following:

Corollary 2.

This is the corollary to Theorem 4 for $\eta =0$. For $\tilde{Y},\delta ,\gamma ,$ and α, as defined in Theorem 4, we have the following:

$${\int }_{\tilde{Y}}{|\det (\tilde{Y}{\tilde{Y}}^{*})|}^{\gamma }{\mathrm{e}}^{-\alpha {[\mathrm{tr}(\tilde{Y}{\tilde{Y}}^{*})]}^{\delta }}\mathrm{d}\tilde{Y}=\frac{{\pi }^{pq}{\tilde{\Gamma }}_p(\gamma +q)\Gamma (\frac{p}{\delta }(\gamma +q))}{\delta {\tilde{\Gamma }}_p(q)\Gamma (p(\gamma +q)){\alpha }^{\frac{p}{\delta }(\gamma +q)}}$$

for $\Re (\gamma )>-q+p-1,\delta >0,$ and $\alpha >0$.

Corollary 3.

This is the corollary to Theorem 3 for $\eta =0,\delta =1$. For $Y,\gamma ,$ and α, as defined in Theorem 3, we have the following:

$${\int }_Y{|Y{Y}^{\prime }|}^{\gamma }{\mathrm{e}}^{-\alpha [\mathrm{tr}(Y{Y}^{\prime })]}\mathrm{d}Y=\frac{{\pi }^{\frac{pq}{2}}}{{\Gamma }_p(\frac{q}{2})}\frac{{\Gamma }_p(\gamma +\frac{q}{2})}{{\alpha }^{p(\gamma +\frac{q}{2})}},\Re (\gamma )>-\frac{q}{2}+\frac{p-1}{2},\alpha >0,p\le q.$$

The corresponding result in the complex domain is the following.

Corollary 4.

This is the corollary to Theorem 4 for $\eta =0,\delta =1$. For $\tilde{Y},\gamma ,$ and α, as defined in Theorem 4, we have the following:

$${\int }_{\tilde{Y}}{|\det (\tilde{Y}{\tilde{Y}}^{*})|}^{\gamma }{\mathrm{e}}^{-\alpha [\mathrm{tr}(\tilde{Y}{\tilde{Y}}^{*})]}\mathrm{d}\tilde{Y}=\frac{{\pi }^{pq}}{{\tilde{\Gamma }}_p(q)}\frac{{\tilde{\Gamma }}_p(\gamma +q)}{{\alpha }^{p(\gamma +q)}}$$

for $\Re (\gamma )>-q+p-1$ and $\alpha >0$.

Theorem 5.

Let $U=(u_{ij})>O$ be a $p\times p$ real positive definite matrix with $p(p+1)/2$ functionally independent real scalar variables $u_{ij}$. Then, the following integral over $U>O$ is equivalent to the integral over Y, where Y is a $p\times q,p\le q$ matrix of rank p with $pq$ distinct real scalar elements. Then, we have the following:

$${\int }_{U>O}{|U|}^{\gamma +\frac{q}{2}-\frac{p+1}{2}}{[\mathrm{tr}(U)]}^{\eta }{\mathrm{e}}^{-\alpha {[\mathrm{tr}(U)]}^{\delta }}\mathrm{d}U$$

(26)

$$=\frac{{\Gamma }_p(\frac{q}{2})}{{\pi }^{\frac{pq}{2}}}{\int }_Y{|Y{Y}^{\prime }|}^{\gamma }{[\mathrm{tr}(Y{Y}^{\prime })]}^{\eta }{\mathrm{e}}^{-\alpha {[\mathrm{tr}(Y{Y}^{\prime })]}^{\delta }}\mathrm{d}Y$$

(27)

for $\alpha >0,\delta >0,\Re (\eta )>0,$ and $\Re (\gamma )>-\frac{q}{2}+\frac{p-1}{2}$.

This result enables us to go back and forth from a real full-rank rectangular matrix to a real positive definite matrix. The proof is straightforward. Let $Y{Y}^{\prime }=G$. Then, $G=G^{\prime }>O$. Then, from Lemma 3, $\mathrm{d}Y=\frac{{\pi }^{\frac{pq}{2}}}{{\Gamma }_p(\frac{q}{2})}{|G|}^{\frac{q}{2}-\frac{p+1}{2}}$, which establishes the result. The corresponding result in the complex domain is the following.

Theorem 6.

Let the $p\times p$ matrix in the complex domain $\tilde{U}=({\tilde{u}}_{ij})={\tilde{U}}^{*}>O$ be a Hermitian positive definite, where the $p(p+1)/2$ distinct scalar complex variables are the elements ${\tilde{u}}_{ij}$. Then, the following integral over $\tilde{U}$ is equivalent to the integral over $\tilde{Y}$, where $\tilde{Y}$ is a $p\times q,p\le q$ matrix in the complex domain of rank p with distinct $pq$ complex variables as elements. Then, we have the following:

$$\begin{array}{cc}\hfill & {\int }_{\tilde{U}>O}{|\det (\tilde{U})|}^{\gamma +q-p}{[\mathrm{tr}(\tilde{U})]}^{\eta }{\mathrm{e}}^{-\alpha {[\mathrm{tr}(\tilde{U})]}^{\delta }}\mathrm{d}U\hfill \\ \hfill & =\frac{{\tilde{\Gamma }}_p(q)}{{\pi }^{pq}}{\int }_{\tilde{Y}}{|\det (\tilde{Y}{\tilde{Y}}^{*})|}^{\gamma }{[\mathrm{tr}(\tilde{Y}{\tilde{Y}}^{*})]}^{\eta }{\mathrm{e}}^{-\alpha {[\mathrm{tr}(\tilde{Y}{\tilde{Y}}^{*})]}^{\delta }}\mathrm{d}\tilde{Y}\hfill \end{array}$$

for $\alpha >0,\Re (\eta )>0,$ and $\Re (\gamma )>-q+p-1$.

The proof is parallel to that in the real case. Here, we use Lemma 3 in the complex case; that is the only difference.

3. Some Integrals Involving Type 2 Beta Forms

Let X be a $p\times 1$ vector in the real domain with distinct scalar variables as elements. Then, we have the following multivariate type 2 beta density:

Theorem 7.

$${\int }_X\frac{{(X^{\prime }X)}^{\delta }}{{[1+\alpha {(X^{\prime }X)}^{\eta }]}^{\gamma }}\mathrm{d}X=\frac{{\pi }^{\frac{p}{2}}}{\Gamma (\frac{p}{2})}\frac{\Gamma [\frac{1}{\eta }(\delta +\frac{p}{2})]\Gamma (\gamma -\frac{1}{\eta }(\delta +\frac{p}{2}))}{\eta \Gamma (\gamma ){\alpha }^{\frac{1}{\eta }(\delta +\frac{p}{2})}}$$

(28)

for $\Re (\gamma )>\frac{1}{\eta }(\Re (\delta )+\frac{p}{2}),\eta >0,\Re (\delta )>-\frac{p}{2},$ and $\alpha >0$.

This result is easily seen from Lemma 3. Note that $\mathrm{d}X=\frac{{\pi }^{\frac{p}{2}}}{\Gamma (\frac{p}{2})}{u}^{\frac{p}{2}-1}\mathrm{d}u$. Let $v=u^{\eta }$. Now, integrate out by using a scalar variable type 2 beta integral to establish the result. The integrand of the left side divided by the right side gives a statistical density. One can generalize the result in Theorem 7 by replacing $X^{\prime }X$ with ${(X-\mu )}^{\prime }A(X-\mu )$, where $\mu =E[X],A>O$ is a constant positive definite matrix. Then, the only change is that the right side of (28) is multiplied by ${|A|}^{-\frac{1}{2}}$, which is the positive definite square root of the positive definite matrix $A>O$. The result corresponding to Theorem 7 in the complex domain will be stated next without any proof, because the derivation is parallel to that in the real case.

Theorem 8.

Let $\tilde{X}$ be a $p\times 1$ vector of p distinct scalar complex variables as elements. Then, we have the following multivariate type 2 beta density:

$${\int }_{\tilde{X}}\frac{{[{\tilde{X}}^{*}\tilde{X}]}^{\delta }}{{(1+\alpha {[{\tilde{X}}^{*}\tilde{X}]}^{\eta })}^{\gamma }}\mathrm{d}\tilde{X}=\frac{{\pi }^p}{\Gamma (p)}\frac{\Gamma (\frac{1}{\eta }(\delta +p))\Gamma (\gamma -\frac{1}{\eta }(\delta +p))}{\eta \Gamma (\gamma ){\alpha }^{\frac{1}{\eta }(\delta +p)}}$$

for $\Re (\gamma )>\frac{1}{\eta }(\Re (\delta )+p),\eta >0,\Re (\delta )>-p,$ and $\alpha >0$.

Theorem 8 follows from the complex part of Lemma 3, and then we integrate out the real scalar variable by using a type 2 beta integral. Now, we consider the evaluation of a rectangular matrix-variate type 2 beta integral in the real case.

Theorem 9.

Let $X=(x_{ij})$ be a $p\times q,p\le q$ matrix in the real domain of rank p with $pq$ distinct real scalar variables as the elements $x_{ij}$. Then, we have the following:

$${\int }_X\frac{{[\mathrm{tr}(X{X}^{\prime })]}^{\delta }}{{[1+\alpha {[\mathrm{tr}(X{X}^{\prime })]}^{\eta }]}^{\gamma }}\mathrm{d}X=\frac{{\pi }^{\frac{pq}{2}}}{\Gamma (\frac{pq}{2})}\frac{\Gamma [\frac{1}{\eta }(\delta +\frac{pq}{2})]\Gamma (\gamma -\frac{1}{\eta }(\delta +\frac{pq}{2}))}{\eta \Gamma (\gamma ){\alpha }^{\frac{1}{\eta }(\delta +\frac{pq}{2})}}$$

(29)

for $\Re (\delta )>-\frac{pq}{2},\Re (\gamma )>\frac{1}{\eta }(\Re (\delta )+\frac{pq}{2}),\eta >0,$ and $\alpha >0$.

Note that $\mathrm{tr}(X{X}^{\prime })$ is the sum of squares of $pq$ elements. Then, from Lemma 3, the result follows, or, from Theorem 7, the result follows by replacing $\frac{p}{2}$ with $\frac{pq}{2}$. A more general situation is to replace $X{X}^{\prime }$ with $A^{\frac{1}{2}}(X-M)B{(X-M)}^{\prime }{A}^{\frac{1}{2}}$, where $M=E[X],A>O,$ and $B>O$, where A is a $p\times p$ constant positive definite matrix, and B is a $q\times q$ constant positive definite matrix. In this case, the only change will be to multiply the right side of (29) by ${|A|}^{-\frac{q}{2}}{|B|}^{-\frac{p}{2}}$. The result corresponding to Theorem 9 in the complex domain will be stated next without the details, because the details, as well as the generalizations, are parallel to those in the real case.

Theorem 10.

Let $\tilde{X}=({\tilde{x}}_{ij})$ be a $p\times q,p\le q$ matrix in the complex domain of rank p with $pq$ distinct scalar complex variables as elements. Then, we have the following:

$${\int }_{\tilde{X}}\frac{{[\mathrm{tr}(\tilde{X}{\tilde{X}}^{*})]}^{\delta }}{{(1+\alpha {[\mathrm{tr}(\tilde{X}{\tilde{X}}^{*})]}^{\eta })}^{\gamma }}\mathrm{d}\tilde{X}=\frac{{\pi }^{pq}}{\Gamma (pq)}\frac{\Gamma [\frac{1}{\eta }(\delta +pq)]\Gamma (\gamma -\frac{1}{\eta }(\delta +pq))}{\eta \Gamma (\gamma ){\alpha }^{\frac{1}{\eta }(\delta +pq)}}$$

for $\Re (\delta )>-pq,\Re (\gamma )>\frac{1}{\eta }(\Re (\delta )+pq),\eta >0,$ and $\alpha >0$.

This theorem follows from the complex part of Lemma 3; then, we convert the term of the form $u^{\eta }=v$ and integrate out v by using a type 2 beta integral to establish Theorem 10.

The next result involves a determinant.

Theorem 11.

Let $X=(x_{ij})$ be a $p\times q,p\le q$ real matrix of rank p, where the elements $x_{ij}$ are distinct real scalar variables. Then, we have the following:

$${\int }_X\frac{|X{X}^{\prime }{|}^{\gamma }}{{[1+\alpha {[\mathrm{tr}(X{X}^{\prime })]}^{\delta }]}^{\rho }}\mathrm{d}X=\frac{\Gamma (\frac{p}{\delta }(\gamma +\frac{q}{2}))\Gamma (\rho -\frac{p}{\delta }(\gamma +\frac{q}{2}))}{\delta {\alpha }^{\frac{p}{\delta }(\gamma +\frac{q}{2})}}\frac{{\pi }^{\frac{pq}{2}}}{{\Gamma }_p(\frac{q}{2})}\frac{{\Gamma }_p(\gamma +\frac{q}{2})}{\Gamma (p(\gamma +\frac{q}{2}))}$$

for $\alpha >0,\delta >0,\Re (\rho )>\frac{p}{\delta }(\gamma +\frac{q}{2}),\Re (\gamma )>-\frac{q}{2}+\frac{p-1}{2}$.

Proof. 

Let $X=T{U}_1$, where T is a lower triangular matrix, and $U_1$ is a semiorthonormal matrix $U_1{U}_1^{\prime }=I_p$, and let T and $U_1$ be uniquely chosen. Then, from Lemma 4, we have the following:

$$\mathrm{d}X=\frac{{\pi }^{\frac{pq}{2}}}{{\Gamma }_p(\frac{q}{2})}\{\prod _{j=1}^p|t_{jj}{|}^{q-j}\}\mathrm{d}T,X{X}^{\prime }=T{T}^{\prime }.$$

Note that $|X{X}^{\prime }|=|T{T}^{\prime }|={\prod }_{j=1}^p{t}_{jj}^2$ and $\mathrm{tr}(T{T}^{\prime })={\sum }_{j=1}^p{t}_{jj}^2+{\sum }_{i>j}{t}_{ij}^2=$ the sum of the squares of $p(p+1)/2$ real scalar variables. Now, apply a polar coordinate transformation to these $p(p+1)/2$ variables $t_{ij}$, and we have the following:

$$[t_{11},t_{22},\dots ,t_{pp},t_{21},\dots ,t_{pp-1}]\to [r,{\theta }_1,\dots ,{\theta }_{k-1}],k=p(p+1)/2.$$

By collecting all of the factors containing r, we have ${(r^2)}^{p(\gamma +\frac{q}{2})-\frac{1}{2}}$. Now, by integrating over r, we have

$${\int }_0^{\infty }{(r^2)}^{p(\gamma +\frac{q}{2})-\frac{1}{2}}{[1+\alpha {(r^2)}^{\delta }]}^{-\rho }\mathrm{d}r=\frac{\Gamma [\frac{p}{\delta }(\gamma +\frac{q}{2})]\Gamma (\rho -\frac{p}{\delta }(\gamma +\frac{q}{2}))}{2\delta \Gamma (\rho ){\alpha }^{\frac{p}{\delta }(\gamma +\frac{q}{2})}}$$

for $\delta >0,\Re (\gamma )>-\frac{q}{2}+\frac{p-1}{2},\Re (\rho )>\frac{p}{\delta }(\Re (\gamma )+\frac{q}{2}),$ and $\alpha >0$. From Theorem 1, the integral over the ${\theta }_j$s gives $2{\Gamma }_p(\gamma +\frac{q}{2})/\Gamma (p(\gamma +\frac{q}{2}))$, and from the transformation of $X{X}^{\prime }$ to $T{T}^{\prime }$, we have ${\pi }^{\frac{pq}{2}}/{\Gamma }_p(\frac{q}{2})$. Hence, the product of these three quantities establishes the theorem. The result corresponding to Theorem 11 in the complex domain will be given next without the proof. The proof goes parallel to that in the real case. In this connection, observe the derivation of the sine and cosine factors in the complex case given earlier; the number of terms in $\mathrm{tr}(\tilde{T}{\tilde{T}}^{*})$ will be $p^2$ in the complex case, and it is $p(p+1)/2$ in the real case. □

Theorem 12.

Let $\tilde{X}=({\tilde{x}}_{ij})$ be a $p\times q,p\le q$ matrix in the complex domain of rank p with $pq$ distinct scalar complex variables as elements. Then, we have the following:

$${\int }_{\tilde{X}}\frac{|\det (\tilde{X}{\tilde{X}}^{*}){|}^{\gamma }}{{[1+\alpha {[\mathrm{tr}(\tilde{X}{\tilde{X}}^{*})]}^{\delta }]}^{\rho }}\mathrm{d}\tilde{X}=\frac{\Gamma (\frac{p}{\delta }(\gamma +q))\Gamma (\rho -\frac{p}{\delta }(\gamma +q))}{\delta {\alpha }^{\frac{p}{\delta }(\gamma +q)}}\frac{{\pi }^{pq}{\tilde{\Gamma }}_p(\gamma +q)}{{\tilde{\Gamma }}_p(q)\Gamma (p(\gamma +q))}$$

for $\alpha >0,\delta >0,\Re (\rho )>\frac{p}{\delta }(\gamma +q),$ and $\Re (\gamma )>-q+p-1$, where ${\tilde{\Gamma }}_p(·)$ is the complex matrix-variate gamma.

Theorem 12 is established by first going through the complex part of Lemma 4, then converting the $t_{ij}$s into a $p^2$ polar coordinates, and then using Theorem 8. The next result will involve a determinant and a trace raised to an arbitrary power in the numerator.

Theorem 13.

Let $X,p,q,\delta$, and ρ be as defined in Theorem 11. Let $\Re (\eta )>0$. Then, we have the following:

$$\begin{array}{cc}\hfill {\int }_X\frac{|X{X}^{\prime }{|}^{\gamma }{[\mathrm{tr}(X{X}^{\prime })]}^{\eta }}{{[1+\alpha [\mathrm{tr}{(X{X}^{\prime })}^{\delta }]]}^{\rho }}\mathrm{d}X& =\frac{{\pi }^{\frac{pq}{2}}{\Gamma }_p(\gamma +\frac{q}{2})}{{\Gamma }_p(\frac{q}{2})\Gamma (p(\gamma +\frac{q}{2}))}\hfill \\ \hfill & \times \frac{\Gamma (\frac{1}{\delta }(p(\gamma +\frac{q}{2})+\eta ))\Gamma (\rho -\frac{1}{\delta }(p(\gamma +\frac{q}{2})+\eta ))}{\delta {\alpha }^{\frac{1}{\delta }(p(\gamma +\frac{q}{2})+\eta )}}\hfill \end{array}$$

for $\Re (\rho )>\frac{1}{\delta }(p(\gamma +\frac{q}{2})+\eta ),\delta >0,\Re (\gamma )>-\frac{q}{2}+\frac{p-1}{2},\Re (\eta )>0,$ and $\alpha >0$.

Here, the proof is the same as that of Theorem 11; the only difference is that the polar variable r will have an additional exponent $\eta$. The corresponding result in the complex domain is the following:

Theorem 14.

Let $\tilde{X},p,q,\delta$, and ρ be as defined in Theorem 12. Let $\Re (\eta )>0$. Then, we have the following:

$$\begin{array}{cc}\hfill {\int }_{\tilde{X}}\frac{|\det (\tilde{X}{\tilde{X}}^{*}){|}^{\gamma }{[\mathrm{tr}(\tilde{X}{\tilde{X}}^{*})]}^{\eta }}{{[1+\alpha {[\mathrm{tr}(\tilde{X}{\tilde{X}}^{*})]}^{\delta }]}^{\rho }}& =\frac{{\pi }^{pq}{\tilde{\Gamma }}_p(\gamma +q)}{{\tilde{\Gamma }}_p(q)\Gamma (p(\gamma +q))}\hfill \\ \hfill & \times \frac{\Gamma (\frac{1}{\delta }(p(\gamma +q)+\eta ))\Gamma (\rho -\frac{1}{\delta }(p(\gamma +q)+\eta ))}{\delta {\alpha }^{\frac{1}{\delta }(p(\gamma +q)+\eta )}}\hfill \end{array}$$

for $\Re (\rho )>\frac{1}{\delta }(p(\Re (\gamma )+q)+\Re (\eta )),\delta >0,\Re (\eta )>0,\alpha >0,$ and $\Re (\gamma )>-q+p-1$.

Here, the proof is the same as that of Theorem 12; the only difference is that the polar variable r will have an additional exponent $\eta$.

Remark 2.

Theorems 9–13 can be generalized by replacing $X{X}^{\prime }$ with $A^{\frac{1}{2}}(X-M)B{(X-M)}^{\prime }{A}^{\frac{1}{2}}$, where $M=E[X],$ and $A>O$ and $B>O$ are, respectively, $p\times p$ and $q\times q$ constant positive definite matrices, and $A^{\frac{1}{2}}$ is the positive definite square root of $A>O$. The only change is that the right sides of the equations are to be multiplied by ${|A|}^{-\frac{q}{2}}{|B|}^{-\frac{p}{2}}$. If $\alpha \to 0$, and if $\rho =\frac{\xi }{\alpha }$ and $\xi >0$, then we have the following:

$${[1+\alpha {[\mathrm{tr}(X{X}^{\prime })]}^{\delta }]}^{-\frac{\xi }{\alpha }}\to {\mathrm{e}}^{-\xi {[\mathrm{tr}(X{X}^{\prime })]}^{\delta }}$$

Thus, the integrands in Theorems 9–13 are Mathai’s pathway models. Then, the models in Section 3 go to the models in Section 2. In the complex case, one can have the corresponding generalizations. $\tilde{X}{\tilde{X}}^{*}$ may be replaced by $A^{\frac{1}{2}}(\tilde{X}-\tilde{M}B{(\tilde{X}-\tilde{M})}^{*}{A}^{\frac{1}{2}}$, where $\tilde{M}=E[\tilde{X}],A=A^{*}>O,$ and $B=B^{*}>O$ (both are Hermitian positive definites), and $A^{\frac{1}{2}}$ denotes the Hermitian positive definite square root of the Hermitian positive definite matrix A. Now, we consider a generalized logistic format or an exponentiated beta form of a matrix-variate integral.

Theorem 15.

Let X be a $p\times q,p\le q$ matrix of rank p, where the $pq$ elements are distinct real scalar variables. Then, we have the following:

$$\begin{array}{cc}\hfill {\int }_X\frac{|X{X}^{\prime }{|}^{\gamma }{[\mathrm{tr}(X{X}^{\prime })]}^{\eta }{\mathrm{e}}^{-\alpha {[\mathrm{tr}(X{X}^{\prime })]}^{\delta }}}{{(1+a{\mathrm{e}}^{-{[\mathrm{tr}(X{X}^{\prime })]}^{\delta }})}^{\alpha +\beta }}\mathrm{d}X& =\frac{{\pi }^{\frac{pq}{2}}{\Gamma }_p(\gamma +\frac{q}{2})}{{\Gamma }_p(\frac{q}{2})}\frac{\Gamma [\frac{1}{\delta }(p(\gamma +\frac{q}{2})+\eta )]}{\delta \Gamma (p(\gamma +\frac{q}{2}))}\hfill \\ \hfill & \times \zeta [\{(\frac{1}{\delta }(p(\gamma +\frac{q}{2})+\eta ),\alpha )\}:\alpha +\beta ;;-a]\hfill \end{array}$$

for $\delta >0,0<a<1,\Re (\alpha )>0,\Re (\beta )>0,\Re (\gamma )>-\frac{q}{2}+\frac{p-1}{2},$ and $\Re (\eta )>0$, and $\zeta [·]$ is a Mathai’s extended zeta function, which is also given in Remark 3 below.

Proof. 

Since $0<a{\mathrm{e}}^{-{[\mathrm{tr}(X{X}^{\prime })]}^{\delta }}<1$, one can use a binomial expansion as follows:

$${[1+a{\mathrm{e}}^{-{[\mathrm{tr}(X{X}^{\prime })]}^{\delta }}]}^{-(\alpha +\beta )}=\sum _{k=0}^{\infty }{(\alpha +\beta )}_k\frac{{(-a)}^k}{k!}{\mathrm{e}}^{-k{[\mathrm{tr}(X{X}^{\prime })]}^{\delta }}$$

and ${\mathrm{e}}^{-\alpha {[\mathrm{tr}(X{X}^{\prime })]}^{\delta }}{\mathrm{e}}^{-k{[\mathrm{tr}(X{X}^{\prime })]}^{\delta }}={\mathrm{e}}^{-(\alpha +k){[\mathrm{tr}(X{X}^{\prime })]}^{\delta }}$. Now, apply Theorem 3 to see the result. □

Remark 3.

For the real scalar variable x, the logistic density is the following:

$$\frac{{\mathrm{e}}^{-x}}{{(1+{\mathrm{e}}^{-x})}^2}=\frac{{\mathrm{e}}^x}{{(1+{\mathrm{e}}^x)}^2},-\infty <x<\infty .$$

(30)

This density behaves similarly to a standard Gaussian density, but the logistic density has a thicker tail compared with that of the standard Gaussian. Hence, in many industrial applications, a logistic model is preferred to a standard Gaussian model. A generalized logistic density was introduced as the following:

$$f(x)=\frac{\Gamma (\alpha +\beta )}{\Gamma (\alpha )\Gamma (\beta )}\frac{{\mathrm{e}}^{-\alpha x}}{{(1+{\mathrm{e}}^{-x})}^{\alpha +\beta }},-\infty <x<\infty .$$

(31)

This model in (31) is more viable, and asymmetric situations can also be covered under this generalized model (31) compared to (30). Note that, for $\alpha =1=\beta$ in (31), we have (30). Hence, the matrix-variate analogues of the logistic-based models are connected to the generalized logistic density in the (31) above. The model in (31) is the exponentiated type 2 beta density. Make the transformation $x={\mathrm{e}}^{-y}$ in a type 2 beta density to go to model (31). Matrix-variate versions of logistic-based densities usually end up in an extended form of a generalized zeta function. The zeta function $\zeta (\rho )$ and the generalized zeta function $\zeta (\rho ,\alpha )$, available in the literature, are the following:

$$z(\rho )=\sum _{j=1}^{\infty }\frac{1}{k^{\rho }},\Re (\rho )>1;\zeta (\rho ,\alpha )=\sum _{j=0}^{\infty }\frac{1}{{(\alpha +k)}^{\rho }},\Re (\rho )>1,\alpha \ne 0,-1,\dots$$

(32)

The extended zeta function is the following:

$$\begin{array}{cc}\hfill {\zeta }_{p,q}^r(x)& =\zeta [\{(m_1,{\alpha }_1),\dots ,(m_r,{\alpha }_r)\}:a_1,\dots ,a_p;b_1,\dots ,b_q;x]\hfill \\ \hfill & =\sum _{k=0}^{\infty }\frac{1}{{({\alpha }_1+k)}^{m_1}...{({\alpha }_r+k)}^{m_r}}\frac{{(a_1)}_k\dots {(a_p)}_k}{{(b_1)}_k\dots {(b_q)}_k}\frac{x^k}{k!}\hfill \end{array}$$

for ${\sum }_{j=1}^r{m}_r>1,{\alpha }_j\ne 0,-1,\dots ,j=1,\dots ,r;b_j\ne 0,-1,\dots ,j=1,\dots ,q;q\ge p$ or $p=q+1$, and $|x|<1$, where, for example, ${(a)}_k$ is the Pochhamer symbol defined as ${(a)}_k=a(a+1)\dots (a+k-1),a\ne 0,{(a)}_0=1$. The result corresponding to Theorem 15 in the complex case is the following: The details of the derivations are parallel to those in the real case and, hence, are not given here.

Theorem 16.

Let $\tilde{X}$ be a $p\times q,p\le q$ matrix in the complex domain of rank p with $pq$ distinct scalar complex variables as elements. Then, we have the following:

$$\begin{array}{cc}\hfill & {\int }_{\tilde{X}}\frac{|\det (\tilde{X}{\tilde{X}}^{*}){|}^{\gamma }{[\mathrm{tr}(\tilde{X}{\tilde{X}}^{*})]}^{\eta }{\mathrm{e}}^{-\alpha {[\mathrm{tr}(\tilde{X}{\tilde{X}}^{*})]}^{\delta }}}{{(1+a{\mathrm{e}}^{-{[\mathrm{tr}(\tilde{X}{\tilde{X}}^{*})]}^{\delta }})}^{\alpha +\beta }}\mathrm{d}\tilde{X}\hfill \\ \hfill & =\frac{{\pi }^{pq}{\tilde{\Gamma }}_p(\gamma +q)\Gamma (\frac{1}{\delta }(p(\gamma +q)+\eta ))}{\delta {\tilde{\Gamma }}_p(q)\Gamma (p(\gamma +q))}\zeta [\{((\frac{1}{\delta }(p(\gamma +q)+\eta )),\alpha )\}:\alpha +\beta ;;-a]\hfill \end{array}$$

for $\delta >0,0<a<1,\Re (\alpha )>0,\Re (\beta )>0,\Re (\gamma )>-q+p-1,$ and $\Re (\eta )>0$, and $\zeta [·]$ is defined in Theorem 15 above.

The proof is the same as that in Theorem 14, except that the $\alpha$ in the exponent is now replaced by $\alpha +k$, and then the result is interpreted in terms of an extended zeta function.

4. Matrix-Variate Type 1 Beta Forms

Let X be a $p\times 1$ vector of distinct real scalar variables. Consider the following multivariate function:

$$f_1(X)=c_1{[X^{\prime }X]}^{\gamma }{[1-a{(X^{\prime }X)}^{\delta }]}^{\beta -1},\Re (\beta )>0,a>0,a{(X^{\prime }X)}^{\delta }<1$$

That is, X is confined to the interior of the p-dimensional sphere of radius ${(\frac{1}{a})}^{\frac{1}{\delta }}$, and $f_1(X)$ is assumed to be zero outside of this sphere. If $c_1$ is the normalizing constant so that $f_1(X)$ is a density, let us compute $c_1$. Let $u=X^{\prime }X\Rightarrow \mathrm{d}X=\frac{{\pi }^{\frac{p}{2}}}{\Gamma (\frac{p}{2})}{u}^{\frac{p}{2}-1}\mathrm{d}u$ by Lemma 3. Let $v=u^{\delta }\Rightarrow \mathrm{d}u=\frac{1}{\delta }{v}^{\frac{1}{\delta }-1}\mathrm{d}v$. Then, we have the following:

$$\begin{array}{cc}\hfill 1& ={\int }_X{f}_1(X)\mathrm{d}X=c_1\frac{{\pi }^{\frac{p}{2}}}{\Gamma (\frac{p}{2})}{\int }_u{u}^{\gamma +\frac{p}{2}-1}{(1-a{u}^{\delta })}^{\beta -1}\mathrm{d}u\hfill \\ \hfill & =c_1\frac{{\pi }^{\frac{p}{2}}}{\Gamma (\frac{p}{2})}\frac{1}{\delta }{\int }_v{v}^{\frac{1}{\delta }(\gamma +\frac{p}{2})-1}{(1-av)}^{\beta -1}\mathrm{d}v\hfill \\ \hfill & =c_1\frac{{\pi }^{\frac{p}{2}}}{\Gamma (\frac{p}{2})}\frac{1}{\delta }\frac{\Gamma (\frac{1}{\delta }(\gamma +\frac{p}{2}))\Gamma (\beta )a^{-\frac{1}{\delta }(\gamma +\frac{p}{2})}}{\Gamma (\beta +\frac{1}{\delta }(\gamma +\frac{p}{2}))},\Re (\gamma )>-\frac{p}{2}\Rightarrow \hfill \\ \hfill c_1& =\frac{\delta \Gamma (\frac{p}{2})\Gamma (\beta +\frac{1}{\delta }(\gamma +\frac{p}{2}))a^{\frac{1}{\delta }(\gamma +\frac{p}{2})}}{{\pi }^{\frac{p}{2}}\Gamma (\beta )\Gamma (\frac{1}{\delta }(\gamma +\frac{p}{2}))},\Re (\gamma )>-\frac{p}{2}\hfill \end{array}$$

(33)

for $0<a<1,\delta >0,$ and $\Re (\beta )>0$. Note that $f_1(X)$ is also the density connected with type 1 beta-distributed isotropic random points in geometrical probability problems. The corresponding format in Section 3 is associated with type 2 beta-distributed random points. A more general model is available by replacing $X^{\prime }X$ with ${(X-\mu )}^{\prime }A(X-\mu ),$ where $\mu =E[X],$ and $A>O$ is a $p\times p$ real constant positive definite matrix. The only change will be that the normalizing constant $c_1$ will be multiplied by ${|A|}^{\frac{1}{2}}$, and the structure of the function remains the same.

Theorem 17.

Let X be a $p\times 1$ real vector of distinct real scalar variables as elements. Consider the quadratic form ${(X-\mu )}^{\prime }A(X-\mu ),E[X]=\mu ,$ and $A>O$, where A is a $p\times p$ constant positive definite matrix. Let $0<a<1,\delta >0,0<a{[{(X-\mu )}^{\prime }A(X-\mu )]}^{\delta }<1,\Re (\beta )>0,$ and $\Re (\eta )>0$. Then, in:

$$f_1(X)=c_1{[\mathrm{tr}({(X-\mu )}^{\prime }A(X-\mu ))]}^{\eta }{[1-a{[\mathrm{tr}({(X-\mu )}^{\prime }A(X-\mu ))]}^{\delta }]}^{\beta -1},$$

for $a{[\mathrm{tr}({(X-\mu )}^{\prime }A(X-\mu ))]}^{\delta }<1$, and $f_1(X)=0$ elsewhere, is given by the following:

$$c_1=\frac{{\delta |A|}^{\frac{1}{2}}\Gamma (\frac{p}{2})a^{\frac{1}{\delta }(\gamma +\frac{p}{2})}\Gamma (\beta +\frac{1}{\delta }(\gamma +\frac{p}{2}))}{{\pi }^{\frac{p}{2}}\Gamma (\beta )\Gamma (\frac{1}{\delta }(\gamma +\frac{p}{2}))},\Re (\gamma )>-\frac{p}{2}.$$

The density and the normalizing constant in the complex case, corresponding to Theorem 17, are the following. The evaluation of the normalizing constant is parallel to that in the real case, and, hence, only the results are given here.

Theorem 18.

Let $\tilde{X}$ be a $p\times 1$ vector in the complex domain with distinct scalar complex variables as elements. Let $\tilde{u}={(\tilde{X}-\tilde{\mu })}^{*}A(\tilde{X}-\tilde{\mu })$, which is a Hermitian form where $\tilde{\mu }=E[X],A=A^{*}>O$ is a constant Hermitian positive definite matrix. Note that the Hermitian form $\tilde{u}$ is real, and, hence, the following function $f_{1c}(\tilde{X})$ is real-valued and is a density when ${\tilde{c}}_1$ is the normalizing constant there. Let $0<a<1,\delta >0,0<\tilde{u}<1,\Re (\beta )>0,$ and $\Re (\eta )>0$. Then, the density $f_{1c}(\tilde{X})$ and the normalizing constant ${\tilde{c}}_1$ are the following:

$$f_{1c}(\tilde{X})={\tilde{c}}_1{[u]}^{\eta }{[1-a{u}^{\delta }]}^{\beta -1},u={(\tilde{X}-\tilde{\mu })}^{*}A(\tilde{X}-\tilde{\mu }),a<u^{\delta }<1$$

(34)

and $f_{1c}(\tilde{X})=0$ elsewhere; we then have the following:

$${\tilde{c}}_1=\frac{\delta |\det (A)|\Gamma (p)a^{\frac{1}{\delta }(\gamma +p)}\Gamma (\beta +\frac{1}{\delta }(\gamma +p))}{{\pi }^p\Gamma (\beta )\Gamma (\frac{1}{\delta }(\gamma +p))},\Re (\gamma )>-p.$$

Now, consider $X=(x_{ij})$ to be a $p\times q,p\le q$ matrix of rank p, where the $pq$ elements $x_{ij}$ are distinct real scalar variables. Consider the following model:

$$f_2(X)=c_2{[\mathrm{tr}(X{X}^{\prime })]}^{\eta }{[1-a{[\mathrm{tr}(X{X}^{\prime })]}^{\delta }]}^{\beta -1},$$

(35)

for $a>0,\delta >0,\Re (\beta )>0,\Re (\eta )>-\frac{pq}{2},a{[\mathrm{tr}(X{X}^{\prime })]}^{\delta }<1$, and $f_2(X)=0$ outside of this sphere. Note that $\mathrm{tr}(X{X}^{\prime })=$ the sum of the squares of $pq$ real scalar variables here, and $u=\mathrm{tr}(X{X}^{\prime })\Rightarrow \mathrm{d}X=\frac{{\pi }^{\frac{pq}{2}}}{\Gamma (\frac{pq}{2})}{u}^{\frac{pq}{2}-1}\mathrm{d}u.$ Then, proceeding as in the derivation of $c_1$ in $f_1(X)$, or in Theorem 17, we have the following.

Theorem 19.

Let $f_2(X)$ be as defined in (35) for X, which is a real $p\times q,p\le q$ matrix of rank p; then, the normalizing constant $c_2$ in $f_2(X)$ is given by the following:

$$c_2=\frac{\Gamma (\frac{pq}{2})}{{\pi }^{\frac{pq}{2}}}\frac{\delta \Gamma (\beta +\frac{1}{\delta }(\eta +\frac{pq}{2}))a^{\frac{1}{\delta }(\eta +\frac{pq}{2})}}{\Gamma (\beta )\Gamma (\frac{1}{\delta }(\gamma +\frac{pq}{2}))}$$

(36)

for $\Re (\beta )>0,a>0,\delta >0,$ and $\Re (\eta )>-\frac{pq}{2}$.

In the corresponding complex case, the result is the following.

Theorem 20.

Let $\tilde{X}$ be a $p\times q,p\le q$ matrix in the complex domain of rank p, where the $pq$ elements are distinct scalar complex variables. Then, the following function $f_{2c}(\tilde{X})$ is a density:

$$f_{2c}(\tilde{X})={\tilde{c}}_2{[\mathrm{tr}(\tilde{X}{\tilde{X}}^{*})]}^{\eta }{[1-a{[\mathrm{tr}(\tilde{X}{\tilde{X}}^{*})]}^{\delta }]}^{\beta -1},0<a{[\mathrm{tr}(\tilde{X}{\tilde{X}}^{*})]}^{\delta }<1$$

(37)

for $a>0,\delta >0,\Re (\eta )>-pq,$ and $\Re (\beta )>0$, and $f_{2c}(\tilde{X})=0$ elsewhere, where

$${\tilde{c}}_2=\frac{\Gamma (pq)\delta \Gamma (\beta +\frac{1}{\delta }(\gamma +pq))a^{\frac{1}{\delta }(\gamma +pq)}}{{\pi }^{pq}\Gamma (\beta )\Gamma (\frac{1}{\delta }(\gamma +pq))}$$

(38)

for $\Re (\beta )>0,0<a<1,\delta >0,$ and $\Re (\eta )>-pq$.

A more general model is available in the real case by replacing $X{X}^{\prime }$ with $A^{\frac{1}{2}}(X-M)B{(X-M)}^{\prime }{A}^{\frac{1}{2}}$, where $A>O$ and $B>O$ are constant $p\times p$ and $q\times q$ matrices, respectively, and $M=E[X]$. Consider the transformation $Y=A^{\frac{1}{2}}(X-M)B^{\frac{1}{2}}\Rightarrow \mathrm{d}Y={|A|}^{\frac{q}{2}}|B^{\frac{p}{2}}\mathrm{d}X$ by Lemma 1. Then, the density of Y is the same as the $f_2(X)$ of (36). Hence, the only change will be that the normalizing constant in (36) is multiplied by ${|A|}^{\frac{q}{2}}{|B|}^{\frac{p}{2}}$. Therefore, this case is not listed here separately. A similar comment holds in the complex case as well. In the complex case, the multiplicative factor is ${|\det (A)|}^q{|\det (B)|}^p$.

Remark 4.

From the model in Theorem 19, one can easily evaluate the density of $[\mathrm{tr}(X{X}^{\prime })]$ or in the general case the density of $[\mathrm{tr}(A^{\frac{1}{2}}(X-M)B{(X-M)}^{\prime }{A}^{\frac{1}{2}})]$ from the normalizing constant $c_2$. Upon treating $c_2=c_2(\eta )$, we have $E[{[\mathrm{tr}(X{X}^{\prime })]}^h]=\frac{c_2(\eta )}{c_2(\eta +h)}$ for an arbitrary h. Hence, from the inverse Mellin transform, one has the density of $\mathrm{tr}(X{X}^{\prime })$ or that in the general case. The same comment holds for Theorem 17 as well. Similar comments hold in the complex case as well.

If the multiplicative factor ${[\mathrm{tr}(X{X}^{\prime })]}^{\eta }$ in the real case is replaced by a determinant $|X{X}^{\prime }{|}^{\gamma }$, let us see what happens to such a model. Again, let X be a $p\times q,p\le q$ matrix of rank p with distinct $pq$ real scalar variables as elements. Consider the following model:

$$f_3(X)=c_3{|X{X}^{\prime }|}^{\gamma }{[1-a{[\mathrm{tr}(X{X}^{\prime })]}^{\delta }]}^{\beta -1},$$

(39)

for $\Re (\beta )>0,0<a<1,a{[\mathrm{tr}(X{X}^{\prime })]}^{\delta }<1,\delta >0,$ and $\Re (\gamma )>-\frac{q}{2}+\frac{p-1}{2}$, and $f_3(X)=0$ elsewhere. Then, we have the following result.

Theorem 21.

Let X and the parameters be as defined in (39). Then, we have the following:

$$c_3=\frac{{\Gamma }_p(\frac{q}{2})\delta a^{\frac{p}{\delta }(\gamma +\frac{q}{2})}\Gamma (\beta +\frac{p}{\delta }(\gamma +\frac{q}{2}))\Gamma (p(\gamma +\frac{q}{2}))}{{\pi }^{\frac{pq}{2}}\Gamma (\beta )\Gamma (\frac{p}{\delta }(\gamma +\frac{q}{2})){\Gamma }_p(\gamma +\frac{q}{2})}$$

for $\Re (\gamma )>-\frac{q}{2}+\frac{p-1}{2},0<a<1,\delta >0,\Re (\beta )>0,$ and $p\le q.$

Proof. 

Let $X=T{U}_1$, where $T=(t_{ij})$ is a lower triangular matrix, and $U_1$ is a semiorthonormal matrix $U_1{U}_1^{\prime }=I_p$, where both T and $U_1$ are uniquely chosen. Then, from Lemma 4, after integrating out the differential element corresponding to the semiorthonormal matrix $U_1$, one has the following relationship:

$$\mathrm{d}X=\frac{{\pi }^{\frac{pq}{2}}}{{\Gamma }_p(\frac{q}{2})}[\prod _{j=1}^p|t_{jj}{|}^{q-j}]\mathrm{d}T,X{X}^{\prime }=T{T}^{\prime }.$$

(40)

Note that $|X{X}^{\prime }|=|T{T}^{\prime }|={\prod }_{j=1}^p{t}_{jj}^2$, and

$$|T{T}^{\prime }{|}^{\gamma }\{\prod _{j=1}^p|t_{jj}{|}^{q-j}\}=\prod _{j=1}^p{(t_{jj}^2)}^{\gamma +\frac{q}{2}-\frac{j}{2}},\mathrm{tr}(T{T}^{\prime })=\sum _{j=1}^p{t}_{jj}^2+\sum _{i>j}{t}_{ij}^2$$

(41)

where in ${\sum }_{i>j}{t}_{ij}^2$, there are $p(p-1)/2$ terms. Consider a polar coordinate transformation on all of these $p(p+1)/2$ terms’ $t_{ij}$s. This results in $\{t_{11},t_{22},\dots ,t_{pp},t_{21},\dots ,t_{pp-1}\}\to \{r,{\theta }_1,\dots ,{\theta }_{k-1}\},k=p(p+1)/2$. Then, the Jacobian element is already discussed in the proof of Theorem 1. r has exponent $k-1$ in the Jacobian element. Then, by collecting all of the factors containing r in the transformed $f_3(X)$, we have the following:

$${(r^2)}^{p(\gamma +\frac{q}{2})-\frac{1}{2}}{[1-a{(r^2)}^{\delta }]}^{\beta -1}$$

and the integral over r gives us the following:

$${\int }_0^{\infty }{(r^2)}^{p(\gamma +\frac{q}{2})-\frac{1}{2}}{(1-a{(r^2)}^{\delta })}^{\beta -1}\mathrm{d}r=\frac{1}{2\delta }{a}^{-\frac{p}{\delta }(\gamma +\frac{q}{2})}\frac{\Gamma (\frac{p}{\delta }(\gamma +\frac{q}{2}))\Gamma (\beta )}{\Gamma (\beta +\frac{p}{\delta }(\gamma +\frac{q}{2}))}$$

(42)

for $\Re (\beta )>0,a>0,\delta >0,$ and $\Re (\gamma )>-\frac{q}{2}$. The integral over all of the sine and cosine products is available from the proof of Theorem 1, which is $2{\Gamma }_p(\gamma +\frac{q}{2})/\Gamma (p(\gamma +\frac{q}{2}))$ for $\Re (\gamma )>-\frac{q}{2}+\frac{p-1}{2}$. Taking the product with that in (42) establishes the theorem. In the complex case, the density and the normalizing constant are the following:

$$f_{3c}(\tilde{X}={\tilde{c}}_3{|\det (\tilde{X}{\tilde{X}}^{*})|}^{\gamma }{[1-a{[\mathrm{tr}(\tilde{X}{\tilde{X}}^{*})]}^{\delta }]}^{\beta -1}$$

(43)

for an $\tilde{X}$$p\times q,p\le q$ matrix of rank p in the complex domain with distinct $pq$ complex scalar variables as elements such that $a{[\mathrm{tr}(\tilde{X}{\tilde{X}}^{*})]}^{\delta }<1,a>0,\delta >0,$ and $\Re (\gamma )>-q+p-1$, and $f_{3c}(\tilde{X})=0$ elsewhere. Then, the normalizing constant ${\tilde{c}}_3$ is available from the following theorem. □

Theorem 22.

For $\tilde{X},p,q,\delta ,a,$ and γ as defined in (43) and following through the derivation parallel to that in the real case, the normalizing constant ${\tilde{c}}_3$ is the following:

$${\tilde{c}}_3=\frac{{\tilde{\Gamma }}_p(q)\delta a^{\frac{p}{\delta }(\gamma +q)}\Gamma (\beta +\frac{p}{\delta }(\gamma +q))\Gamma (p(\gamma +q))}{{\pi }^{pq}\Gamma (\beta )\Gamma (\frac{p}{\delta }(\gamma +q)){\tilde{\Gamma }}_p(\gamma +q)}$$

for $\Re (\gamma )>-q+p-1,a>0,\delta >0,\Re (\beta )>0,$ and $p\le q$.

Theorem 22 is established by going parallel to the proof in Theorem 21, namely, by going through the complex part of Lemma 4, then converting to $p^2$ polar coordinates, and then using Theorem 2 to evaluate the integral over the sine and cosine product. As explained in Remark 4, the arbitrary moments and exact density of $|X{X}^{\prime }|$ or its general form $|A^{\frac{1}{2}}(X-M)B{(X-M)}^{\prime }{A}^{\frac{1}{2}}|$ in the real case are available from the normalizing constant $c_3$. The corresponding comment holds in the complex case as well.

Note that a more general model in the real case is also available by replacing $X{X}^{\prime }$ with $A^{\frac{1}{2}}(X-M)B{(X-M)}^{\prime }{A}^{\frac{1}{2}}$ as mentioned before. The only change will be that the normalizing constant will be multiplied by ${|A|}^{\frac{q}{2}}{|B|}^{\frac{p}{2}}$. Hence, this general case is not listed here separately. In the complex case, the multiplicative factor is ${|\det (A)|}^q{|\det (B)|}^p$. A more general case is available by introducing another factor containing a trace into $f_3(X)$. Consider the following model in the real case:

$$f_4(X)=c_4{|X{X}^{\prime }|}^{\gamma }{[\mathrm{tr}(X{X}^{\prime })]}^{\eta }{[1-a{[\mathrm{tr}(X{X}^{\prime })]}^{\delta }]}^{\beta -1},a>0,\delta >0,a{[\mathrm{tr}(X{X}^{\prime })]}^{\delta }<1$$

(44)

for $\Re (\beta )>0,\Re (\eta )>0,$ and $\Re (\gamma )>-\frac{q}{2}+\frac{p-1}{2};X$ is a $p\times q,p\le q$ matrix of rank p, $a>0,a{[\mathrm{tr}(X{X}^{\prime })]}^{\delta }<1,$ and $\delta >0$, and $f_4(X)=0$ is outside of this sphere. Proceeding exactly as in the proof of Theorem 21, we have the following result.

Theorem 23.

Let $X,p,q,\delta ,\eta ,a,$ and γ be as defined in (44), and $\Re (\eta )>0$. Then, the normalizing constant $c_4$ is given by the following:

$$c_4=\frac{{\Gamma }_p(\frac{q}{2})}{{\pi }^{\frac{pq}{2}}}\frac{\delta a^{\frac{1}{\delta }(p(\gamma +\frac{q}{2})+\eta )}\Gamma (\beta +\frac{1}{\delta }(p(\gamma +\frac{q}{2})+\eta ))\Gamma (p(\gamma +\frac{q}{2}))}{\Gamma (\beta )\Gamma (\frac{1}{\delta }(p(\gamma +\frac{q}{2})+\eta )){\Gamma }_p(\gamma +\frac{q}{2})}.$$

The model corresponding to the one in (44) in the complex case is the following, where $\tilde{X}$ is a $p\times q,p\le q$ matrix in the complex domain of rank p with $pq$ distinct scalar complex variables as elements such that $a{[\mathrm{tr}(\tilde{X}{\tilde{X}}^{*})]}^{\delta }<1,a>0,$ and $\delta >0$; the parameters are such that $\Re (\gamma )>-q+p-1,\Re (\eta )>0,$ and $\Re (\beta )>0$. Then, we have the following:

$$f_{4c}(\tilde{X})={\tilde{c}}_4{|\det (\tilde{X}{\tilde{X}}^{*})|}^{\gamma }{[\mathrm{tr}(\tilde{X}{\tilde{X}}^{*})]}^{\eta }{[1-a{[\mathrm{tr}(\tilde{X}{\tilde{X}}^{*})]}^{\delta }]}^{\beta -1}$$

(45)

and $f_{4c}(\tilde{X})=0$ elsewhere. By using the steps parallel to those in the derivation of the normalizing constant $c_4$ in the real case, one can see that the normalizing constant in the complex case is given in the following theorem.

Theorem 24.

For $\tilde{X},a,\delta ,\eta ,\delta ,$ and γ as defined in (45), the normalizing constant ${\tilde{c}}_4$ is the following:

$${\tilde{c}}_4=\frac{{\tilde{\Gamma }}_p(q)\delta a^{\frac{1}{\delta }(p(\gamma +q)+\eta )}\Gamma (\beta +\frac{1}{\delta }(p(\gamma +q)+\eta ))\Gamma (p(\gamma +q))}{{\pi }^{pq}\Gamma (\beta )\Gamma (\frac{1}{\delta }(p(\gamma +q)+\eta )){\tilde{\Gamma }}_p(\gamma +q)}.$$

The proof is the same as that in Theorem 22, except that here the polar variable r will have an additional exponent $\eta$. Again, a more general model in the real case is available by replacing $X{X}^{\prime }$ with $A^{\frac{1}{2}}(X-M)B{(X-M)}^{\prime }{A}^{\frac{1}{2}},A>O,B>O,$ and $M=E[X]$. In addition, from the structure of $f_4(X)$, it is clear that the exact density and arbitrary moments of the determinant $|XX|$ or $|A^{\frac{1}{2}}(X-M)B{(X-M)}^{\prime }{A}^{\frac{1}{2}}|$ are available by replacing the parameter $\gamma$ with $\gamma +h$ and then taking the ratio of the normalizing constant $c_4$ as explained before. Similarly, the exact density and arbitrary moments of $[\mathrm{tr}(X{X}^{\prime })]$ or its general form are available by replacing $\eta$ with $\eta +h$ and taking the ratio of the normalizing constant $c_4$. Similar comments hold in the complex domain as well: the multiplicative factor will be ${|\det (A)|}^q{|\det (B)|}^p$ in the complex case.

Now, we will consider an exponentiated type 1 beta-type model. Again, let X be a $p\times q,p\le q$ matrix of rank p with $pq$ distinct real scalar variables as elements. Consider the following model:

$$f_5(X)=c_5{|X{X}^{\prime }|}^{\gamma }{[\mathrm{tr}(X{X}^{\prime })]}^{\eta }{\mathrm{e}}^{-\alpha {[\mathrm{tr}(X{X}^{\prime })]}^{\delta }}{[1-a{\mathrm{e}}^{-{[\mathrm{tr}(X{X}^{\prime })]}^{\delta }}]}^{\beta -1}$$

(46)

for $a>0,a{[\mathrm{tr}(X{X}^{\prime })]}^{\delta }<1,\delta >0,\Re (\beta )>0,\Re (\alpha )>0,\Re (\eta )>0,$ and $\Re (\gamma )>-\frac{q}{2}+\frac{p-1}{2},$ and $f_5(X)=0$ outside of the sphere. Then, we have the following result.

Theorem 25.

Let X and the parameters be as defined in $f_5(X)$. Then, the normalizing constant $c_5$ is given by the following:

$$\begin{array}{cc}\hfill c_5& =\frac{{\Gamma }_p(\frac{q}{2})\delta \Gamma (p(\gamma +\frac{q}{2}))}{{\pi }^{\frac{pq}{2}}\Gamma (\frac{1}{\delta }(p(\gamma +\frac{q}{2})+\eta )){\Gamma }_p(\gamma +\frac{q}{2})}\hfill \\ \hfill & \times {[\zeta [\{((\frac{1}{\delta }(p(\gamma +\frac{q}{2})+\eta )),\alpha )\}:1-\beta ;;a]]}^{-1}\hfill \end{array}$$

for $\delta >0,a>0,\Re (\alpha )>0,\Re (\beta )>0,\Re (\eta )>0,$ and $\Re (\gamma )>-\frac{q}{2}+\frac{p-1}{2}$, where $\zeta (·)$ is the extended zeta function defined in Theorem 15.

The proof is straightforward. Since $0<a{[\mathrm{tr}(X{X}^{\prime })]}^{\delta }<1$, we can use a binomial expansion and write the following:

$${[1-a{[\mathrm{tr}(X{X}^{\prime })]}^{\delta }]}^{\beta -1}=\sum _{k=0}^{\infty }{(1-\beta )}_k\frac{a^k}{k!}{\mathrm{e}}^{-k{[\mathrm{tr}(X{X}^{\prime })]}^{\delta }}$$

(47)

Now, the exponential trace part joins with the exponential trace part remaining in $f_5(X)$ to become ${\mathrm{e}}^{-(\alpha +k){[\mathrm{tr}(X{X}^{\prime })]}^{\delta }}$. Then, one can integrate this out by using Theorem 23 by replacing the $\alpha$ there with $\alpha +k$ and interpreting the result in terms of an extended zeta function; thus, Theorem 25 is established. The corresponding model in the complex domain is the following:

$$f_{5c}(\tilde{X}={\tilde{c}}_5{|\det (\tilde{X}{\tilde{X}}^{*})|}^{\gamma }{[\mathrm{tr}(\tilde{X}{\tilde{X}}^{*})]}^{\eta }{\mathrm{e}}^{-\alpha {[\mathrm{tr}(\tilde{X}{\tilde{X}}^{*})]}^{\delta }}{[1-a{\mathrm{e}}^{-{[\mathrm{tr}(\tilde{X}{\tilde{X}}^{*})]}^{\delta }}]}^{\beta -1}$$

(48)

where $\tilde{X}$ is a $p\times q,p\le q$ matrix of rank p in the complex domain with $pq$ distinct scalar complex variables as elements: $a>0,\delta >0,a[{\mathrm{e}}^{-{[\mathrm{tr}(\tilde{X}{\tilde{X}}^{*})]}^{\delta }}<1,\Re (\eta )>0,\Re (\beta )>0,$ and $\Re (\gamma )>-q+p-1$, and $f_{5c}(\tilde{X})=0$ elsewhere. Then, following through the derivation parallel to that in the real case, the normalizing constant ${\tilde{c}}_5$ is the following.

Theorem 26.

Under the conditions stated in (48), we have the following:

$$\begin{array}{cc}\hfill {\tilde{c}}_5& =\frac{{\tilde{\Gamma }}_p(q)\delta \Gamma (p(\gamma +q))}{{\pi }^{pq}\Gamma (\frac{1}{\delta }(p(\gamma +q)+\eta )){\tilde{\Gamma }}_p(\gamma +q)}\hfill \\ \hfill & \times {[\zeta [\{((\frac{1}{\delta }(p(\gamma +q)+\eta )),\alpha )\}:1-\beta ;;a]]}^{-1}.\hfill \end{array}$$

Here, the proof is parallel to that in the proof of Theorem 25. Since exponentiation is involved, the denominator factor is first expanded, and then the same procedure is used as in the proof of Theorem 24.

5. Concluding Remarks

Special cases of all of the normalizing constants reported in Section 2, Section 3 and Section 4, namely, for the cases $\delta =1$ and the exponent of the trace factor $\eta =0$, are available in the recent book [12]. In the real case, the models in Theorems 7 and 17 for the exponent of the gamma factor $\gamma \ne 0$ and and the exponent of the trace factor $\eta =0$, as well as the corresponding multivariate gamma distributions, are connected to geometrical probability problems. The theory of geometrical probabilities in the complex domain is not yet developed. When such a theory is developed, all of the results in this paper will be applicable there. Various models have been defined in this paper by evaluating the corresponding normalizing constants. In order to limit the size of the paper, we did not delve into some of the properties of these models. Chapter 3 of [1] lists various statistical models that are used in the analysis of PolSAR data. They are the following distributions used in the scalar texture models: (1) K distribution—this is nothing more than a gamma distribution in the real scalar case; (2) Kummer U distribution—this is the distribution of a constant multiple of a real scalar type 2 beta random variable; (3) W distribution—this is the distribution of a constant multiple of a real scalar type 1 beta random variable; (4) M distribution—this is again a type 2 beta distribution; (5) Wishart-generalized gamma distribution—this is a generalized gamma distribution in the real scalar case; and (6) G distribution—this is the Bessel-type or Krätzel-type real scalar model discussed in [13] in connection with the generalized reaction rate probability integral in nuclear reaction rate theory. In the multilook or matrix texture models, the format is $U=A^{\frac{1}{2}}Y$, where $A>O$ (a positive definite) is the texture component, and Y is the freckle component. The use of the Kotz model is mentioned on pages 45–48 of [1]. Thus, all of the materials discussed in the present paper will be useful in PolSAR or similar data analysis.

For further work, one can study the properties of the various models introduced here. In the light of [14], one can look into Bayesian models connected with the various distributions introduced in the present paper. In this case, for example, Y can have the distribution discussed in Theorem 3 and particular cases thereof, and $\tilde{Y}$ can have the distribution in Theorem 4 and particular cases thereof. Then, $A^{\frac{1}{2}}Y$ with $A>O$ will be a scale mixture of Y. If $U=A^{\frac{1}{2}}Y$, then the conditional density of U, given A and denoted by $f(U|A)$, is that of Y, with Y replaced by $A^{-\frac{1}{2}}U$. With any compatible prior distribution for A, denoted by $g(A)$, one has the joint density of U, A is given by $f(U|A)g(A)$, and the unconditional density of U, denoted by $f_u(U)$, is then $f_u(U)={\int }_{A}f(U|A)g(A)\mathrm{d}A$. Then, the Bayes’ analysis is the study of A in the conditional distribution of A, given U. One can look into scalar texture models and matrix texture models in this respect, or one can look at the problem from the point of view of Bayesian analysis. This author is looking into this aspect. One can explore the distributions of quantities such as the trace or determinant in connection with the models in the present paper. Since the models introduced in this paper are functions of the trace and determinant, it is easy to convert the densities as joint densities of the eigenvalues of $Y{Y}^{\prime }$ in the real case and $\tilde{Y}{\tilde{Y}}^{*}$ in the complex domain. Then, one can look into the various models introduced in the current paper as eigenvalue problems, and one can study the properties, connections to other problems, and applications. These are some of the ideas that are open for exploration.

The author received no external funding for this research. The author declares no conflict of interest. The author would like to thank the three reviewers for their comments and suggestions. These comments and suggestions enabled the author to improve the presentation of the material in the paper.