Essential Conditions for the Full Synergy of Probability of Occurrence Distributions

Abstract

In this contribution, we specify the conditions for assuring the validity of the synergy of the distribution of probabilities of occurrence. We also study the subsequent restriction on the maximal extension of the strict concavity region on the parameter space of Sharma–Mittal entropy measures, which has been derived in a previous paper in this journal. The present paper is then a necessary complement to that publication. Some applications of the techniques introduced here are applied to protein domain families (Pfam databases, versions 27.0 and 35.0). The results will show evidence of their usefulness for testing the classification work performed with methods of alignment that are used by expert biologists.

1. Introduction

We have been working since 2015 on the problem of testing the alignment of protein domain families which are proposed by expert biologists and bioinformaticians. We have found that the use of selected entropy measures is very proficient for testing the results published by those professionals and they favour a rigorous ANOVA statistical analysis [1]. In order to reduce the search space for admissible values of entropy measures, we have emphasized the need for work in the region related to strict concavity of these entropies. This study has been undertaken in a previous work, and we present in Section 2 a summary of those developments. In the present work, we aim to complement the results of a previous publication [2], and a subsequent restriction on the parameter space has to be performed in order to guarantee the synergy of the probability distributions to be tested. Non-synergetic distributions are not worthwhile for working because they will not preserve the fundamental property of getting more information of amino acids into t-sets of columns than to sum up the information obtained from individual columns. In Section 3, a brief digression is then made for introducing the Sharma–Mittal class of entropy measures. Section 4 emphasizes the aspects of synergy of the distributions and their consequences for the reduction of the parameter space of Sharma–Mittal entropies. In Section 5, we treat the analysis of the maximal extension of the parameter space, and we repeat the reduction process imposed by the requirement of fully synergetic distributions of Section 4. We conclude the paper in Section 6 by studying the relation of Hölder and generalized Khinchin–Shannon (GKS) inequalities.

2. The Construction of the Probabilistic Space

Let us consider a set of $m_f$ domains ($m_f$ rows) from a chosen family of protein domains. In order to associate a rectangular array with this family, to be taken as its representative in the probabilistic space we are constructing, we specify its number of columns as $n_f=n$. This means that among $m_f$ rows, we disregard all rows such that the number of their amino acids satisfies $n_f<n$ and preserve $m_f^{\prime }$ rows whose number of amino acids satisfies $n_f\le n$, but disregard ($n_f-n$) amino acids in these $m_f^{\prime }$ rows. We then choose m rows from among the $m_f^{\prime }$ rows to obtain $m\times n$ rectangular arrays. There are $m_f^{\prime }!/[m!(m_f^{\prime }-m)!]$ of these $m\times n$ rectangular arrays. Any one of them can be used as a representative of the domain family to be analysed in the statistical procedure to be implemented.

The next step is to assign a joint probability of occurrence of a set of variables $a_1,\dots ,a_t$ in columns $j_1,\dots ,j_t$ to be given by

$$p_{j_1\dots j_t}(a_1,\dots ,a_t)=\frac{n_{j_1\dots j_t}(a_1,\dots ,a_t)}{m},$$

(1)

where $n_{j_1\dots j_t}(a_1,\dots ,a_t)$ stands for the number of occurrences of the set $a_1,\dots ,a_t$ in the t columns of the subarray $m\times t$ of the representative array $m\times n$ ($1\le t\le n$). The symbols $a_1,\dots ,a_t$ will be running over the letters of the one-letter code for the twenty amino acids: $a_j(1\le j\le t)\in$ {A, C, D, E, F, G, H, I, K, L, M, N, P, Q, R, S, T, V, W, Y}.

We then have

$$\sum _{a_1,\dots ,a_t}{n}_{j_1\dots j_t}(a_1,\dots ,a_t)\equiv m.$$

(2)

We also introduce the conditional probabilities of occurrence, which are given implicitly by

$$p_{j_1\dots j_t}(a_1,\dots ,a_t)\equiv p_{j_1\dots j_t}(a_1,\dots ,a_{t-1}|a_t)p_{j_t}\left(a_t\right),$$

(3)

where $p_{j_1\dots j_t}(a_1,\dots ,a_{t-1}|a_t)$ is the probability of occurrence of the amino acids in the columns $j_1,\dots ,j_{t-1}$, if the distribution of amino acids in the $j_t$-th column is known a priori.

The Bayes’ law for probabilities of occurrence [2,3] can be written as

$$\begin{array}{cc}\hfill p_{j_1\dots j_t}(a_1,\dots ,a_t)& \equiv p_{j_1\dots j_t}(a_1,\dots ,a_{t-1}|a_t)p_{j_t}\left(a_t\right)\hfill \\ \hfill & =p_{j_t{j}_1\dots j_{t-1}}\left(a_t\right|a_1,\dots ,a_{t-1})p_{j_1\dots j_{t-1}}(a_1,\dots ,a_{t-1})\hfill \\ \hfill & =p_{j_t{j}_{t-1}{j}_1\dots j_{t-2}}(a_t,a_{t-1}|a_1,\dots ,a_{t-2})p_{j_1\dots j_{t-2}}(a_1,\dots ,a_{t-2})\hfill \\ \hfill & =\dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \hfill \\ \hfill & =p_{j_t\dots j_3{j}_1{j}_2}(a_t,\dots ,a_3|a_1,a_2)p_{j_1{j}_2}(a_1,a_2)\hfill \\ \hfill & =p_{j_t\cdots j_2{j}_1}(a_t,\dots ,a_2|a_1)p_{j_1}\left(a_1\right)\hfill \\ \hfill & =p_{j_t\dots j_1}(a_t,\dots ,a_1).\hfill \end{array}$$

(4)

The equality of the three first right-side members, as well as the equality of the three last ones, does correspond to the application of Bayes’ law [2,3]. The symmetries for the joint probability distribution $p_{j_1\dots j_t}(a_1,\dots ,a_t)$ are due to the ordering of the columns for the distributions of amino acids.

From the ordering $j_1<j_2<\dots <j_t$, the values assumed by the variables $j_1,\dots ,j_t$ are respectively given by

$$\begin{array}{cc}\hfill j_1& =1,2,\dots ,n-t+1\hfill \\ \hfill j_2& =j_1+1,j_1+2,\dots ,n-t+2\hfill \\ \hfill & ⋮,1\le t\le n\hfill \\ \hfill j_{t-1}& =j_{t-2}+1,j_{t-2}+2,\dots ,n-1\hfill \\ \hfill j_t& =j_{t-1}+1,j_{t-1}+2,\dots ,n.\hfill \end{array}$$

(5)

We then have $\left(\genfrac{}{}{0pt}{}{n}{t}\right)=\frac{n!}{t!(n-t)!}$ geometric objects $p_{j_1\dots j_t}(a_1,\dots ,a_t)$ of t columns and ${\left(20\right)}^t$ components each.

3. The Sharma–Mittal Class of Entropy Measures

As emphasized in Ref. [2], the introduction of random variable functions such as entropy measures associated with the probabilities of occurrence, is suitable to provide an analysis of the evolution of these probabilities through the regions of the parameter space of entropies. The class of Sharma–Mittal entropy measures seems to be particularly adapted to this task when related to the occurrence of amino acids in the objects $p_{j_1\dots j_t}(a_1,\dots ,a_t)$. The thermodynamic interpretation of the notion of entropy greatly helps to classify the distribution of its values associated with protein domain databases and to interpret its evolution through the Fokker–Planck equations to be treated in forthcoming articles in this line of research.

The two-parameter Sharma–Mittal class of entropy measures is usually given by

$${\left(SM\right)}_{j_1\dots j_t}^{(r,s)}=\frac{{\left({\alpha }_{j_1\dots j_t}^{\left(s\right)}\right)}^{\frac{1-r}{1-s}}-1}{1-r};{\left(SM\right)}_{j_t}^{(r,s)}=\frac{{\left({\alpha }_{j_t}^{\left(s\right)}\right)}^{\frac{1-r}{1-s}}-1}{1-r},$$

(6)

where

$${\alpha }_{j_1\dots j_t}^{\left(s\right)}=\sum _{a_1,\dots ,a_t}{\left[p_{j_1\dots j_t}(a_1,\dots ,a_t)\right]}^s;{\alpha }_{j_t}^{\left(s\right)}=\sum _{a_t}{\left[p_{j_t}\left(a_t\right)\right]}^s.$$

(7)

The parameters r, s must bound a region corresponding to a strict concavity in the parameter space. A necessary requirement to be satisfied [3] is

$$\frac{{\partial }^2{\left(SM\right)}_{j_1\dots j_t}^{(r,s)}}{\partial {\left(p_{j_1\dots j_t}(a_1,\dots ,a_t)\right)}^2}=s{\left({\alpha }_{j_1\dots j_t}^{\left(s\right)}\right)}^{\frac{s-r}{1-s}}·{\left[p_{j_1\dots j_t}(a_1,\dots ,a_t)\right]}^{s-2}·\left[\frac{s(s-r)}{{(1-s)}^2}{\widehat{p}}_{j_1\dots j_t}(a_1,\dots ,a_t)-1\right]<0,$$

(8)

where ${\widehat{p}}_{j_1\dots j_t}(a_1,\dots ,a_t)$ stands for the escort probability associated with the joint probability $p_{j_1\dots j_t}(a_1,\dots ,a_t)$, or,

$${\widehat{p}}_{j_1\dots j_t}(a_1,\dots ,a_t)=\frac{{\left[p_{j_1\dots j_t}(a_1,\dots ,a_t)\right]}^s}{{\alpha }_{j_1\dots j_t}^{\left(s\right)}};{\widehat{p}}_{j_t}\left(a_t\right)=\frac{{\left[p_{j_t}\left(a_t\right)\right]}^s}{{\alpha }_{j_t}^{\left(s\right)}}.$$

(9)

Equation (8) leads to

$$r\ge s>0.$$

(10)

Some special cases of one-parameter entropies are commonplace in the scientific literature [3,4,5,6,7,8,9]:

The $r=s$ region is the domain of the Havrda–Charvat [6] entropy measure $H_{j_1\dots j_t}^{\left(s\right)}$,

$$H_{j_1\dots j_t}^{\left(s\right)}=\frac{{\alpha }_{j_1\dots j_t}^{\left(s\right)}-1}{1-s}.$$

(11)

The $r=2-s$, $0\le s\le 1$, region will stand for the domain of the Landsberg–Vedral [7] entropy measure, $L_{j_1\dots j_t}^{\left(s\right)}$,

$$L_{j_1\dots j_t}^{\left(s\right)}=\frac{{\alpha }_{j_1\dots j_t}^{\left(s\right)}-1}{(1-s){\alpha }_{j_1\dots j_t}^{\left(s\right)}}\equiv \frac{H_{j_1\dots j_t}^{\left(s\right)}}{{\alpha }_{j_1\dots j_t}^{\left(s\right)}}.$$

(12)

The Renyi $R_{j_1\dots j_t}^{\left(s\right)}$ [8] and the “non-extensive” Gaussian [9] $G_{j_1\dots j_t}^{\left(r\right)}$ entropy measures are obtained from limit processes:

$$\begin{array}{cc}\hfill R_{j_1\dots j_t}^{\left(s\right)}& \equiv \underset{r\to 1}{lim}\frac{{\left({\alpha }_{j_1\dots j_t}^{\left(s\right)}\right)}^{\frac{1-r}{1-s}}-1}{1-r}=\underset{r\to 1}{lim}\frac{\frac{\mathrm{d}}{\mathrm{d}r}\left[{\left({\alpha }_{j_1\dots j_t}^{\left(s\right)}\right)}^{\frac{1-r}{1-s}}-1\right]}{\frac{\mathrm{d}}{\mathrm{d}r}(1-r)}=\underset{r\to 1}{lim}\frac{e^{\frac{1-r}{1-s}·log{\alpha }_{j_1\dots j_t}^{\left(s\right)}}·\left(\frac{-1}{1-s}\right)log{\alpha }_{j_1\dots j_t}^s}{-1}\hfill \\ \hfill & =\frac{log{\alpha }_{j_1\dots j_t}^{\left(s\right)}}{1-s}.\hfill \end{array}$$

(13)

$$\begin{array}{cc}\hfill G_{j_1\dots j_t}^{\left(r\right)}& \equiv \underset{s\to 1}{lim}\frac{{\left({\alpha }_{j_1\dots j_t}^{\left(s\right)}\right)}^{\frac{1-r}{1-s}}-1}{1-r}=\frac{1}{1-r}\left[exp\left((1-r)·\underset{s\to 1}{lim}\frac{\frac{\mathrm{d}}{\mathrm{d}s}log{\alpha }_{j_1\dots j_t}^{\left(s\right)}}{\frac{\mathrm{d}}{\mathrm{d}s}(1-s)}\right)-1\right]\hfill \\ \hfill & =\frac{1}{1-r}\left[exp\left(-(1-r)·\underset{s\to 1}{lim}\frac{\frac{\mathrm{d}}{\mathrm{d}s}{\alpha }_{j_1\dots j_t}^{\left(s\right)}}{{\alpha }_{j_1\dots j_t}^{\left(s\right)}}\right)-1\right].\hfill \end{array}$$

After using the definition of ${\alpha }_{j_1\dots j_t}^{\left(s\right)}$, Equation (7), and $\underset{s\to 1}{lim}{\alpha }_{j_1\dots j_t}^{\left(s\right)}=1$ from Equations (1) and (2), we get:

$$G_{j_1\dots j_t}^{\left(r\right)}=\frac{e^{(1-r)S_{j_1\dots j_t}}-1}{1-r},$$

(14)

where $S_{j_1\dots j_t}$ is the Gibbs–Shannon entropy measure

$$S_{j_1\dots j_t}=-\sum _{a_1,\dots ,a_t}{p}_{j_1\dots j_t}(a_1,\dots ,a_t)log{p}_{j_1\dots j_t}(a_1,\dots ,a_t).$$

(15)

The Gibbs–Shannon entropy measure, Equation (15), is also obtained by taking the convenient limits of the special cases of Sharma–Mittal entropies, Equations (11)–(14):

$$\underset{s\to 1}{lim}{H}_{j_1\dots j_t}^{\left(s\right)}=\underset{s\to 1}{lim}{L}_{j_1\dots j_t}^{\left(s\right)}=\underset{s\to 1}{lim}{R}_{j_1\dots j_t}^{\left(s\right)}=\underset{r\to 1}{lim}{G}_{j_1\dots j_t}^{\left(r\right)}=S_{j_1\dots j_t}.$$

(16)

We shall analyse in the next section the structure of the two-parameter space of Sharma–Mittal entropy by taking into consideration these special cases.

We are now reminded that for the limit of Gibbs–Shannon entropy, a conditional entropy measure is defined [3] by

$$S_{j_1\dots j_{t-1}|j_t}=-\sum _{a_1,\dots ,a_t}{p}_{j_t}\left(a_t\right)p_{j_1\dots j_t}(a_1,\dots ,a_{t-1}|a_t)log{p}_{j_1\dots j_t}(a_1,\dots ,a_{t-1}|a_t).$$

(17)

We then have analogously for the conditional Sharma–Mittal entropy measure [3]

$${\left(SM\right)}_{j_1\dots j_{t-1}|j_t}=\frac{{\left({\displaystyle \sum _{a_1,\dots ,a_t}}{\widehat{p}}_{j_t}\left(a_t\right){\left[p_{j_1\dots j_t}(a_1,\dots ,a_{t-1}|a_t)\right]}^s\right)}^{\frac{1-r}{1-s}}-1}{1-r}.$$

(18)

It is easy to show by trivial calculation that, analogously to Equation (16), we will have

$$\underset{s\to 1}{lim}\underset{r\to s}{lim}{\left(SM\right)}_{j_1\dots j_{t-1}|j_t}=\underset{s\to 1}{lim}\underset{r\to 2-s}{lim}{\left(SM\right)}_{j_1\dots j_{t-1}|j_t}=\underset{s\to 1}{lim}\underset{r\to 1}{lim}{\left(SM\right)}_{j_1\dots j_{t-1}|j_t}=\underset{r\to 1}{lim}\underset{s\to 1}{lim}{\left(SM\right)}_{j_1\dots j_{t-1}|j_t}=S_{j_1\dots j_{t-1}|j_t}.$$

(19)

From Equations (6), (7) and (18) and the application of the Bayes’ law, Equation (4), we can write

$${\left(SM\right)}_{j_1\dots j_t}={\left(SM\right)}_{j_t}+{\left(SM\right)}_{j_1\dots j_{t-1}|j_t}+(1-r){\left(SM\right)}_{j_t}{\left(SM\right)}_{j_1\dots j_{t-1}|j_t}.$$

(20)

4. Aspects of Synergy and the Reduction of the Parameter Space for Fully Synergetic Distributions

For the Gibbs–Shannon entropy measure, the inequality written by A. Y. Khinchin [3,10] is

$$S_{j_1\dots j_{t-1}|j_t}\le S_{j_1\dots j_{t-1}}.$$

(21)

This inequality would be described by Khinchin as: “On the average, the knowledge a priori of the distribution on the column $j_t$ can only decrease the uncertainty of the distribution on the $j_1,\dots ,j_{t-1}$ columns”. We can write an analogous inequality for the Sharma–Mittal class of entropies

$${\left(SM\right)}_{j_1\dots j_{t-1}|j_t}\le {\left(SM\right)}_{j_1\dots j_{t-1}}.$$

(22)

We then get from Equations (20) and (22)

$${\left(SM\right)}_{j_1\dots j_t}\le {\left(SM\right)}_{j_t}+{\left(SM\right)}_{j_1\dots j_{t-1}}+(1-r){\left(SM\right)}_{j_t}{\left(SM\right)}_{j_1\dots j_{t-1}}.$$

(23)

After iteration of this equation, $t\to t-1\to t-2\to \dots$, we can also write

$${\left(SM\right)}_{j_1\dots j_t}\le \frac{{\displaystyle \prod _{l=1}^t}\left[1+(1-r){\left(SM\right)}_{j_l}\right]-1}{1-r}.$$

(24)

The inequalities in (21)–(24) are associated with what are called “synergetic conditions”. In this section, we also derive the fully synergetic conditions as GKS inequalities.

After using Equations (7) and (9) in Equation (23), we get

$$\frac{{\left({\alpha }_{j_1\dots j_t}^{\left(s\right)}\right)}^{\frac{1-r}{1-s}}}{1-r}\le \frac{{\left({\alpha }_{j_t}^{\left(s\right)}\right)}^{\frac{1-r}{1-s}}·{\left({\alpha }_{j_1\dots j_{t-1}}^{\left(s\right)}\right)}^{\frac{1-r}{1-s}}}{1-r},$$

(25)

and after iteration and use of Equation (24)

$$\frac{{\left({\alpha }_{j_1\dots j_t}^{\left(s\right)}\right)}^{\frac{1-r}{1-s}}}{1-r}\le \frac{{\displaystyle \prod _{l=1}^t}{\left({\alpha }_{j_l}^{\left(s\right)}\right)}^{\frac{1-r}{1-s}}}{1-r}.$$

(26)

The hatched region of strict concavity in the parameter space of Sharma–Mittal entropies, $C=\left\{\right(s,r\left)\right|r\ge s>0\}$, is depicted in Figure 1. The special cases corresponding to Havrda–Charvat’s ($r=s$), Landsberg–Vedral’s ($r=2-s$), Renyi’s ($r=1$), and “non-extensive” Gaussian’s ($s=1$) entropies are also represented.

We can identify three subregions in Figure 1. They will correspond to

$${\mathrm{R}}_{\mathrm{I}}=\left\{(s,r)\right|1>r\ge s>0\}\Rightarrow {\alpha }_{j_1\dots j_t}^{(s<1)}\le \prod _{l=1}^t{\alpha }_{j_t}^{(s<1)},$$

(27)

$${\mathrm{R}}_{\mathrm{II}}=\left\{(s,r)\right|r\ge s>1\}\Rightarrow {\alpha }_{j_1\dots j_t}^{(s>1)}\ge \prod _{l=1}^t{\alpha }_{j_t}^{(s>1)},$$

(28)

$${\mathrm{R}}_{\mathrm{III}}=\left\{(s,r)\right|r>1>s>0\}\Rightarrow {\alpha }_{j_1\dots j_t}^{(s<1)}\le \prod _{l=1}^t{\alpha }_{j_t}^{(s<1)}$$

(29)

where the ordering of $\alpha$-symbols has been obtained from Equation (26). The subregions R${}_{\mathrm{I}}$ and R${}_{\mathrm{III}}$ are what we call fully synergetic subregions, and the corresponding inequalities are the GKS inequalities [2].

The subregions R${}_{\mathrm{I}}$, R${}_{\mathrm{II}}$, and R${}_{\mathrm{III}}$ are depicted in Figure 2a–c, respectively. The union of subregions R${}_{\mathrm{I}}$ and R${}_{\mathrm{III}}$ is the fully synergetic Khinchin–Shannon restriction to be imposed on the strict concavity region of Figure 1 and it is depicted in Figure 2d below.

5. The Maximal Extension of the Parameter Space and Its Reduction for Fully Synergetic Distribution

In Figure 1 and Figure 2d, we have depicted the structure of the strict concavity region for Sharma–Mittal entropy measures and its reduction to a subregion by the application of the requirement of fully synergetic distributions, respectively. Our analysis has used a coarse-grained approach to concavity given by Equations (8) and (10). We now introduce some necessary refinements for characterizing the probability of occurrence in subarrays of m rows and t columns, $m\times t$. For t columns, there are ${\left(20\right)}^t$ possibilities of occurrence of amino acids, which could be a large number, but we could count not individual amino acids, but groups of t-sets of amino acids ($\mu$-groups) which appear on the m rows of the $m\times t$ array. We characterize these $\mu$-groups by $\mu =1,\dots ,m$, from all equal $\mu$-groups ($\mu =1$) to m different $\mu$-groups ($\mu =m$). We also call $q_{\mu }$, the number of equal t-sets of a given $\mu$-group.

In Equation (2), the sum is over all the amino acids that make up the geometric object defined in Equation (1), the probability of occurrence. We can now perform the sum over $\mu$-groups and write

$$\sum _{a_1^{q_{\mu }},\dots ,a_t^{q_{\mu }}}{p}_{j_1\dots j_t}\left(a_1^{q_{\mu }},\dots ,a_t^{q_{\mu }}\right)=\sum _{a_1^{q_{\mu }},\dots ,a_t^{q_{\mu }}}\frac{n_{j_1\dots j_t}\left(a_1^{q_{\mu }},\dots ,a_t^{q_{\mu }}\right)}{m}=\sum _{\mu =1}^m\frac{q_{\mu }}{m}=1,$$

(30)

where $\left(a_1^{q_{\mu }},\dots ,a_t^{q_{\mu }}\right)$ are the t-sets of a $\mu$-group. We also have from Equation (7)

$${\sum }_{a_1^{q_{\mu }},\dots ,a_t^{q_{\mu }}}{\left[p_{j_1\dots j_t}\left(a_1^{q_{\mu }},\dots ,a_t^{q_{\mu }}\right)\right]}^s={\sum }_{a_1^{q_{\mu }},\dots ,a_t^{q_{\mu }}}{\left[\frac{n_{j_1\dots j_t}\left(a_1^{q_{\mu }},\dots ,a_t^{q_{\mu }}\right)}{m}\right]}^s={\sum }_{\mu =1}^m{\left(\frac{q_{\mu }}{m}\right)}^s={\alpha }_{j_1\dots j_t}^{\left(s\right)}.$$

(31)

From Equations (30) and (31), we can now proceed to the calculation of the Hessian matrix for Sharma–Mittal entropy measures. We have for the first derivative of ${\left(SM\right)}_{j_1\dots j_t}$

$$\frac{\partial {\left(SM\right)}_{j_1\dots j_t}}{\partial p_{j_1\dots j_t}\left(a_1^{q_{\mu }},\dots ,a_t^{q_{\mu }}\right)}=\frac{s}{1-s}{\left({\alpha }_{j_1\dots j_t}^{\left(s\right)}\right)}^{\frac{s-r}{1-s}}{\left[p_{j_1\dots j_t}\left(a_1^{q_{\mu }},\dots ,a_t^{q_{\mu }}\right)\right]}^{s-1}.$$

(32)

We then have for a generic element of the Hessian matrix [2]

$$\begin{array}{cc}\hfill H_{q_{\mu }{q}_{\nu }}& =\frac{{\partial }^2{\left(SM\right)}_{j_1\dots j_t}}{\partial p_{j_1\dots j_t}\left(a_1^{q_{\mu }},\dots ,a_t^{q_{\mu }}\right)\partial p_{j_1\dots j_t}\left(a_1^{q_{\nu }},\dots ,a_t^{q_{\nu }}\right)}\hfill \\ \hfill & =s{\left({\alpha }_{j_1\dots j_t}^{\left(s\right)}\right)}^{\frac{s-r}{1-s}}{\left[p_{j_1\dots j_t}\left(a_1^{q_{\mu }},\dots ,a_t^{q_{\mu }}\right)\right]}^{s-2}\left[\frac{s(s-r)}{{(1-s)}^2}\frac{p_{j_1\dots j_t}\left(a_1^{q_{\mu }},\dots ,a_t^{q_{\mu }}\right)}{p_{j_1\dots j_t}\left(a_1^{q_{\nu }},\dots ,a_t^{q_{\nu }}\right)}·{\widehat{p}}_{j_1\dots j_t}\left(a_1^{q_{\nu }},\dots ,a_t^{q_{\nu }}\right)-{\delta }_{\mu \nu }\right],\hfill \end{array}$$

(33)

where ${\widehat{p}}_{j_1\dots j_t}\left(a_1^{q_{\mu }},\dots ,a_t^{q_{\mu }}\right)$ is the escort probability associated to $p_{j_1\dots j_t}\left(a_1^{q_{\mu }},\dots ,a_t^{q_{\mu }}\right)$, or

$${\widehat{p}}_{j_1\dots j_t}\left(a_1^{q_{\mu }},\dots ,a_t^{q_{\mu }}\right)\equiv \frac{{\left[p_{j_1\dots j_t}\left(a_1^{q_{\mu }},\dots ,a_t^{q_{\mu }}\right)\right]}^s}{{\displaystyle \sum _{b_1^{q_{\nu }},\dots ,b_t^{q_{\nu }}}}{p}_{j_1\dots j_t}\left(b_1^{q_{\nu }},\dots ,b_t^{q_{\nu }}\right)}.$$

(34)

The principal minors are given by

$$\begin{array}{cc}\hfill det{H}_{q_{\mu }{q}_{\nu }}(\mu ,\nu =1,\dots ,k)={(-1)}^{k-1}{s}^k{\left({\alpha }_{j_1\dots j_t}^{\left(s\right)}\right)}^{\frac{k(s-r)}{1-s}}{\left[\prod _{\mu =1}^k{p}_{j_1\dots j_t}\left(a_1^{q_{\mu }},\dots ,a_t^{q_{\mu }}\right)\right]}^{s-2}& \hfill \\ \hfill ·\left[\frac{s(s-r)}{{(1-s)}^2}\sum _{\mu =1}^k{\widehat{p}}_{j_1\dots j_t}\left(a_1^{q_{\mu }},\dots ,a_t^{q_{\mu }}\right)-1\right],k=1,\dots ,m,& \hfill \end{array}$$

(35)

and we have

$$\sum _{\mu =1}^k{\widehat{p}}_{j_1\dots j_t}\left(a_1^{q_{\mu }},\dots ,a_t^{q_{\mu }}\right)=\sum _{\mu =1}^k\frac{{\left[p_{j_1\dots j_t}\left(a_1^{q_{\mu }},\dots ,a_t^{q_{\mu }}\right)\right]}^s}{{\displaystyle \sum _{\mu =1}^m}{\left(\frac{q_{\mu }}{m}\right)}^s}=\frac{{\displaystyle \sum _{\mu =1}^k}{\left(\frac{q_{\mu }}{m}\right)}^s}{{\displaystyle \sum _{\mu =1}^m}{\left(\frac{q_{\mu }}{m}\right)}^s}\equiv {\sigma }_k\left(s\right)$$

(36)

according to Equation (31).

From Equations (35) and (36), the requirement of strict concavity will lead to

$$\frac{s(s-r)}{{(1-s)}^2}{\sigma }_k\left(s\right)-1<0.$$

(37)

We then have

$$det{H}_{q_{\mu }{q}_{\nu }}(\mu ,\nu =1,\dots ,k)<0,k\mathrm{odd};>0,k\mathrm{even}.$$

(38)

This does correspond to the criterion of negative definiteness of the Hessian matrix for strict concavity of multivariate functions [11].

Each k-value is associated with the k-epigraph region, which is the k-extension of the strict concavity region presented in Figure 1. These regions are given by

$$C_{k_{\max }}=\left\{(s,r)\right|r\ge s>0\}\cup \left\{(s,r)|s>r>s-\frac{{(1-s)}^2}{s{\sigma }_k\left(s\right)}\right\},k=1,\dots ,m.$$

(39)

The greatest lower bound of the sequence of k-curves is given by ${\sigma }_m\left(s\right)=1$. We then have

$$r_m\left(s\right)=2-\frac{1}{s}.$$

(40)

We can then write for the maximal extended region of strict concavity

$$C_{\max }=\left\{(s,r)\right|r\ge s>0\}\cup \left\{(s,r)|s>r>2-\frac{1}{s}\right\}.$$

(41)

The region corresponding to Equation (41) is depicted in Figure 3 below.

We are now ready to undertake the application of restrictions for fully synergetic distributions (validity of GKS inequalities) to the maximal strict concavity region of Figure 3.

We start by identifying two regions included in Figure 3. They will be given by

$${\mathrm{R}}_{\mathrm{IV}}=\left\{(s,r)|1>s>r\ge 2-\frac{1}{s}>0\right\}\Rightarrow {\alpha }_{j_1\dots j_t}^{(s<1)}\le \prod _{l=1}^t{\alpha }_{j_t}^{(s<1)},$$

(42)

$${\mathrm{R}}_{\mathrm{V}}=\left\{(s,r)|s>r\ge 2-\frac{1}{s}>1\right\}\Rightarrow {\alpha }_{j_1\dots j_t}^{(s>1)}\ge \prod _{l=1}^t{\alpha }_{j_t}^{(s>1)}.$$

(43)

These regions are depicted in Figure 4a,b, respectively.

In order to find the reduced region corresponding to Figure 3, analogously to what has been done for Figure 1, we also need the subregions R${}_{\mathrm{I}}$, R${}_{\mathrm{III}}$, Equations (27) and (29): the resulting subregion of fully synergetic distributions is given by ${\mathrm{R}}_{\mathrm{IV}}\cup {\mathrm{R}}_{\mathrm{I}}\cup {\mathrm{R}}_{\mathrm{III}}$ and is depicted in Figure 5.

6. Hölder Inequalities and GKS Inequalities: A Possible Conjecture

In this section, we study the relation between GKS inequalities [2] and Hölder inequalities by using examples of distributions obtained from databases of protein domain families. In order to start, some definitions and properties of the probabilistic space are now in order.

Let us first introduce the definition of the conditional probability of occurrence of the escort probability of occurrence [12]. This is a simple application to escort probabilities of Equation (3):

$${\widehat{p}}_{j_1\dots j_t}(a_1,\dots ,a_{t-1}|a_t)=\frac{{\widehat{p}}_{j_1\dots j_t}(a_1,\dots ,a_t)}{{\widehat{p}}_{j_t}\left(a_t\right)}.$$

(44)

From the definitions of escort probabilities, Equation (9), we can write

$${\widehat{p}}_{j_1\dots j_t}(a_1,\dots ,a_t)=\frac{{\left(p_{j_1\dots j_t}(a_1,\dots ,a_t)\right)}^s}{{\displaystyle \sum _{b_1,\dots ,b_t}}{\left(p_{j_1\dots j_t}(b_1,\dots ,b_t)\right)}^s},$$

(45)

and

$${\widehat{p}}_{j_t}\left(a_t\right)=\frac{{\left(p_{j_t}\left(a_t\right)\right)}^s}{{\displaystyle \sum _{b_t}}{\left(p_{j_t}\left(b_t\right)\right)}^s}.$$

(46)

In Equations (44)–(46), the symbols $a_1,\dots ,a_t$; $b_1,\dots ,b_t$ assume the representative letters of the one-letter code for the 20 amino acids, $a_j$; $b_j$ $(1\le j\le t)\in$ {A, C, D, E, F, G, H, I, K, L, M, N, P, Q, R, S, T, V, W, Y}.

After substituting Equations (45) and (46) into Equation (44), we get

$${\widehat{p}}_{j_1\dots j_t}(a_1,\dots ,a_{t-1}|a_t)=\frac{{\left(p_{j_1\dots j_t}(a_1,\dots ,a_{t-1}|a_t)\right)}^s{\left(p_{j_t}\left(a_t\right)\right)}^s}{{\displaystyle \sum _{b_1,\dots ,b_t}}{\left(p_{j_t}\left(b_t\right)\right)}^s{\left(p_{j_1\dots j_t}(b_1,\dots ,b_{t-1}|b_t)\right)}^s·{\widehat{p}}_{j_t}\left(a_t\right)},$$

(47)

and from Equation (46)

$${\widehat{p}}_{j_1\dots j_t}(a_1,\dots ,a_{t-1}|a_t)=\frac{{\left(p_{j_1\dots j_t}(a_1,\dots ,a_{t-1}|a_t)\right)}^s}{{\displaystyle \sum _{b_1,\dots ,b_t}}{\widehat{p}}_{j_t}\left(b_t\right){\left(p_{j_1\dots j_t}(b_1,\dots ,b_{t-1}|b_t)\right)}^s}.$$

(48)

We also write the definition of escort probability of occurrence of the conditional probability of occurrence [12]

$$\widehat{p_{j_1\dots j_t}(a_1,\dots ,a_{t-1}|a_t)}=\frac{{\left(p_{j_1\dots j_t}(a_1,\dots ,a_{t-1}|a_t)\right)}^s}{{\displaystyle \sum _{b_1,\dots ,b_{t-1}}}{\left(p_{j_1\dots j_t}(b_1,\dots ,b_{t-1}|b_t)\right)}^s}.$$

(49)

We can check the definitions of Equations (48) and (49) from the equality of the two escort probabilities with the original conditional probability, for $s=1$

$$s=1\Rightarrow {\widehat{p}}_{j_1\dots j_t}(a_1,\dots ,a_{t-1}|a_t)=\widehat{p_{j_1\dots j_t}(a_1,\dots ,a_{t-1}|a_t)}=p_{j_1\dots j_t}(a_1,\dots ,a_{t-1}|a_t).$$

(50)

We should note that the denominators of the right-hand sides of Equations (48) and (49), or,

$$Z\equiv \sum _{a_1,\dots ,a_t}{\widehat{p}}_{j_t}\left(a_t\right){\left[p_{j_1\dots j_t}(a_1,\dots ,a_{t-1}|a_t)\right]}^s=\frac{{\alpha }_{j_1\dots j_t}^{\left(s\right)}}{{\alpha }_{j_t}^{\left(s\right)}},$$

(51)

and

$$X\left(a_t\right)\equiv \sum _{a_1,\dots ,a_{t-1}}{\left(p_{j_1\dots j_t}(a_1,\dots ,a_{t-1}|a_t)\right)}^s$$

(52)

will be equal if all amino acids in the $j_t$ column are equal. If we have, for instance, the $j_t$ column given by:

$$j_t\leftrightarrow \underset{m}{\underbrace{(\mathrm{A},\mathrm{A},\mathrm{A},\mathrm{A},\dots ,\mathrm{A})}}.$$

(53)

The unit vectors of probabilities ${\widehat{p}}_{j_t}$ and $p_{j_t}$ will also be equal and given by

$${\left({\widehat{p}}_{j_t}\right)}^{\mathsf{T}}={\left(p_{j_t}\right)}^{\mathsf{T}}=\underset{20}{\underbrace{(1,0,0,0,\dots ,0)}}.$$

(54)

This means that for this special case of an event of rare occurrence, we also have the equality of the conditional of the escort probability and the escort probability of the conditional probability, or the left-hand sides of Equations (48) and (49), respectively.

For a $j_t$-column with a generic distribution of amino acids, the denominators Z and $X\left(a_t\right)$ on the right-hand sides of Equations (48) and (49) will no longer be equal. An ordering of these denominators should be decided from the probabilities of amino acid occurrence in a chosen protein domain family.

This study is undertaken with the help of the functions Z and $X\left(a_t\right)$ of Equations (51) and (52) and with the functions J and U, defined below:

$$J\equiv \sum _{a_1,\dots ,a_{t-1}}{\left[\sum _{a_t}{\widehat{p}}_{j_t}\left(a_t\right)p_{j_1\dots j_t}(a_1,\dots ,a_{t-1}|a_t)\right]}^s,$$

(55)

$$U\equiv \sum _{a_1,\dots ,a_{t-1}}{\left(p_{j_1\dots j_{t-1}}(a_1,\dots ,a_{t-1})\right)}^s\equiv {\alpha }_{j_1\dots j_{t-1}}^{\left(s\right)}.$$

(56)

Our method will then be the comparison of pairs of functions in order to proceed with the search for the effect of fully synergetic distributions of amino acids.

There are six comparisons to study:

(I)

$X\left(a_t\right)\gtrless Z$

or

$$\frac{1}{{\left(p_{j_t}\left(a_t\right)\right)}^s}\sum _{a_1,\dots ,a_{t-1}}{\left(p_{j_1\dots j_t}(a_1,\dots ,a_t)\right)}^s\gtrless \frac{{\alpha }_{j_1\dots j_t}^{\left(s\right)}}{{\alpha }_{j_t}^{\left(s\right)}};p_{j_t}\left(a_t\right)\ne 0.$$

(57)

(II)

$X\left(a_t\right)\gtrless J$

or

$$\begin{array}{cc}\hfill \frac{1}{{\left(p_{j_t}\left(a_t\right)\right)}^s}\sum _{a_1,\dots ,a_{t-1}}{\left(p_{j_1\dots j_t}(a_1,\dots ,a_t)\right)}^s\gtrless \frac{1}{{\alpha }_{j_t}^{\left(s\right)}}\sum _{a_1,\dots ,a_{t-1}}{\left[\sum _{a_t}{\left(p_{j_t}\left(a_t\right)\right)}^{s-1}{p}_{j_1\dots j_t}(a_1,\dots ,a_t)\right]}^s;& \\ \hfill p_{j_t}\left(a_t\right)\ne 0.\end{array}$$

(58)

(III)

$X\left(a_t\right)\gtrless U$

or

$$\frac{1}{{\left(p_{j_t}\left(a_t\right)\right)}^s}\sum _{a_1,\dots ,a_{t-1}}{\left(p_{j_1\dots j_t}(a_1,\dots ,a_t)\right)}^s\gtrless {\alpha }_{j_1\dots j_{t-1}}^{\left(s\right)}.$$

(59)

(IV)

$U\gtrless J$

or

$$\sum _{a_1,\dots ,a_{t-1}}{\left(p_{j_1\dots j_{t-1}}(a_1,\dots ,a_{t-1})\right)}^s\equiv {\alpha }_{j_1\dots j_{t-1}}^{\left(s\right)}\gtrless \frac{\mathcal{H}}{{\left({\alpha }_{j_t}^{\left(s\right)}\right)}^s},$$

(60)

where $\mathcal{H}$ is defined by,

$$\mathcal{H}\equiv \sum _{a_1,\dots ,a_{t-1}}{\left[\sum _{a_t}{\left(p_{j_t}\left(a_t\right)\right)}^{s-1}{p}_{j_1\dots j_t}(a_1,\dots ,a_t)\right]}^s.$$

(61)

(V)

$J\gtrless Z$

or

$$\sum _{a_1,\dots ,a_{t-1}}{\left[\sum _{a_t}{\widehat{p}}_{j_t}\left(a_t\right)p_{j_1\dots j_t}(a_1,\dots ,a_{t-1}|a_t)\right]}^s\equiv \frac{\mathcal{H}}{{\left({\alpha }_{j_t}^{\left(s\right)}\right)}^s}\gtrless \frac{{\alpha }_{j_1\dots j_t}^{\left(s\right)}}{{\alpha }_{j_t}^{\left(s\right)}}.$$

(62)

(VI)

$U\gtrless Z$

or

$$\sum _{a_1,\dots ,a_{t-1}}{\left(p_{j_1\dots j_{t-1}}(a_1,\dots ,a_{t-1})\right)}^s\equiv {\alpha }_{j_1\dots j_{t-1}}^{\left(s\right)}\gtrless \frac{{\alpha }_{j_1\dots j_t}^{\left(s\right)}}{{\alpha }_{j_t}^{\left(s\right)}}.$$

(63)

Equations (57)–(59) should be multiplied by ${\left(p_{j_t}\left(a_t\right)\right)}^s$ and after that, each one has to be summed over $a_t$. We then have, respectively,

$${\alpha }_{j_1\dots j_t}^{\left(s\right)}\gtrless {\alpha }_{j_1\dots j_t}^{\left(s\right)},$$

(64)

$${\left({\alpha }_{j_t}^{\left(s\right)}\right)}^{s-1}{\alpha }_{j_1\dots j_t}^{\left(s\right)}\gtrless \mathcal{H},$$

(65)

$${\alpha }_{j_1\dots j_t}^{\left(s\right)}\gtrless {\alpha }_{j_t}^{\left(s\right)}·{\alpha }_{j_1\dots j_{t-1}}^{\left(s\right)}.$$

(66)

Equations (60), (62) and (63) can be written, respectively, as

$${\left({\alpha }_{j_t}^{\left(s\right)}\right)}^s·{\alpha }_{j_1\dots j_{t-1}}^{\left(s\right)}\gtrless \mathcal{H}$$

(67)

$$\mathcal{H}\gtrless {\left({\alpha }_{j_t}^{\left(s\right)}\right)}^{s-1}·{\alpha }_{j_1\dots j_t}^{\left(s\right)},$$

(68)

$${\alpha }_{j_t}^{\left(s\right)}·{\alpha }_{j_1\dots j_{t-1}}^{\left(s\right)}\gtrless {\alpha }_{j_1\dots j_t}^{\left(s\right)},$$

(69)

The Hölder’s inequality as applied to probabilities of occurrence [3] is written as

$$\frac{1}{{\left({\alpha }_{j_t}^{\left(s\right)}\right)}^s}{\left[\sum _{a_t}{\left(p_{j_t}\left(a_t\right)\right)}^{s-1}{p}_{j_1\dots j_t}(a_1,\dots ,a_t)\right]}^s\ge \frac{{\displaystyle \sum _{a_t}}{\left[p_{j_1\dots j_t}(a_1,\dots ,a_t)\right]}^s}{{\alpha }_{j_t}^{\left(s\right)}}.$$

(70)

After multiplying by ${\left({\alpha }_{j_t}^{\left(s\right)}\right)}^s$ and summing over $a_1,\dots ,a_{t-1}$, we get

$$\mathcal{H}\ge {\left({\alpha }_{j_t}^{\left(s\right)}\right)}^{s-1}{\alpha }_{j_1\dots j_t}^{\left(s\right)},s\le 1.$$

(71)

We also define

$$O\equiv {\left({\alpha }_{j_t}^{\left(s\right)}\right)}^{s-1}{\alpha }_{j_1\dots j_t}^{\left(s\right)},$$

(72)

$$B\equiv {\left({\alpha }_{j_t}^{\left(s\right)}\right)}^s{\alpha }_{j_1\dots j_{t-1}}^{\left(s\right)}.$$

(73)

We then summarize the results obtained:

• Equation (64) is only an identity: ${\alpha }_{j_1\dots j_t}^{\left(s\right)}={\alpha }_{j_1\dots j_t}^{\left(s\right)}$.
• Equations (65) and (68) can be ordered by Hölder’s inequality, Equations (70) and (71).
• Equations (66) and (69) can be ordered by GKS inequalities, corresponding to fully synergetic distributions of amino acids, ${\alpha }_{j_1\dots j_t}^{(s<1)}\le {\alpha }_{j_t}^{(s<1)}·{\alpha }_{j_1\dots j_{t-1}}^{(s<1)}$.
• Equation (67) cannot be ordered without additional experimental/phenomenological information on the probabilities of occurrence to be obtained from updated versions of protein domain family databases [13].

We now collect the formulae obtained from the analysis performed on this section. Equations (65) and (68) are ordered by Hölder’s inequality. We write

$$\mathcal{H}-O\ge 0,s<1.$$

(74)

Equations (66) and (69) are ordered by GKS inequality. We write

$$B-O\ge 0,s<1.$$

(75)

After using Equation (73), we can write Equation (67) as

$$B-\mathcal{H}\gtrless 0.$$

(76)

In Figure 6a,b we have depicted the curves corresponding to functions $\mathcal{H}-O$ and $B-O$ for seven 3-sets of contiguous columns and 80 rows, chosen from databases Pfam 27.0 and Pfam 35.0, respectively. There are also inset figures in order to show the curves for $s\ge 1$.

In Figure 7a,b, we do the same for the differences $B-\mathcal{H}$. We emphasize that for the 3-sets such that $B-\mathcal{H}\ge 0$, $0\le s\le 1$, the GKS inequalities $B-O\ge 0$ will result from the validity of Hölder’s inequality. We have worked with the PF01926 protein domain family to perform all the calculations.

7. Concluding Remarks

The first comment we want to make to the present work is about the possibility of working in a region of the parameter space that preserves the strict concavity and the fully synergetic structure of the Sharma–Mittal class of entropy measure distributions to be visited by solutions of a new successful statistical mechanics approach. The usual work with Havrda–Charvat distributions describes the evolution along the boundary ($r=s$) of the region ($r\ge s>0$) that was correctly considered to correspond to strict concavity, but it is also known to be non-synergetic for $s>1$. We now have the opportunity to develop this statistical mechanics approach along an extended boundary, preserving the strict concavity and providing the study of the evolution of fully synergetic entropy distributions. A first sketch of these developments will be presented in a forthcoming publication.

With respect to Figure 6 and Figure 7, we could hypothesize that if the ordering of B and $\mathcal{H}$ could not be obtained, this would be due to the poor alignment of some protein domain families we have been using, but we are not confident enough that we could do this, because we would need much more information “in silico” to be obtained from many other protein domain families. In other words, we expect that a good alignment of a protein domain family will result in the ordering of B and $\mathcal{H}$, but we need to verify this in a large number of families from different Pfam versions before we proceed with a proposal of a method to improve the Pfam database. This looks promising for good scientific work in the line of research we have been aiming to introduce in Ref. [2] and in this contribution.