Entropy

Editorial

Jump to: Research

9 pages, 256 KiB

Open AccessEditorial

Robust Procedures for Estimating and Testing in the Framework of Divergence Measures

by Leandro Pardo and Nirian Martín

Entropy 2021, 23(4), 430; https://doi.org/10.3390/e23040430 - 6 Apr 2021

Cited by 2 | Viewed by 1840

Abstract

The approach for estimating and testing based on divergence measures has become, in the last 30 years, a very popular technique not only in the field of statistics, but also in other areas, such as machine learning, pattern recognition, etc [...] Full article

(This article belongs to the Special Issue Robust Procedures for Estimating and Testing in the Framework of Divergence Measures)

Research

Jump to: Editorial

20 pages, 371 KiB

Open AccessArticle

Rare Event Analysis for Minimum Hellinger Distance Estimators via Large Deviation Theory

by Anand N. Vidyashankar and Jeffrey F. Collamore

Entropy 2021, 23(4), 386; https://doi.org/10.3390/e23040386 - 24 Mar 2021

Cited by 1 | Viewed by 2046

Abstract

Hellinger distance has been widely used to derive objective functions that are alternatives to maximum likelihood methods. While the asymptotic distributions of these estimators have been well investigated, the probabilities of rare events induced by them are largely unknown. In this article, we [...] Read more.

Hellinger distance has been widely used to derive objective functions that are alternatives to maximum likelihood methods. While the asymptotic distributions of these estimators have been well investigated, the probabilities of rare events induced by them are largely unknown. In this article, we analyze these rare event probabilities using large deviation theory under a potential model misspecification, in both one and higher dimensions. We show that these probabilities decay exponentially, characterizing their decay via a “rate function” which is expressed as a convex conjugate of a limiting cumulant generating function. In the analysis of the lower bound, in particular, certain geometric considerations arise that facilitate an explicit representation, also in the case when the limiting generating function is nondifferentiable. Our analysis involves the modulus of continuity properties of the affinity, which may be of independent interest. Full article

(This article belongs to the Special Issue Robust Procedures for Estimating and Testing in the Framework of Divergence Measures)

26 pages, 430 KiB

Open AccessEditor’s ChoiceArticle

Distance-Based Estimation Methods for Models for Discrete and Mixed-Scale Data

by Elisavet M. Sofikitou, Ray Liu, Huipei Wang and Marianthi Markatou

Entropy 2021, 23(1), 107; https://doi.org/10.3390/e23010107 - 14 Jan 2021

Cited by 1 | Viewed by 3148

Abstract

Pearson residuals aid the task of identifying model misspecification because they compare the estimated, using data, model with the model assumed under the null hypothesis. We present different formulations of the Pearson residual system that account for the measurement scale of the data [...] Read more.

Pearson residuals aid the task of identifying model misspecification because they compare the estimated, using data, model with the model assumed under the null hypothesis. We present different formulations of the Pearson residual system that account for the measurement scale of the data and study their properties. We further concentrate on the case of mixed-scale data, that is, data measured in both categorical and interval scale. We study the asymptotic properties and the robustness of minimum disparity estimators obtained in the case of mixed-scale data and exemplify the performance of the methods via simulation. Full article

(This article belongs to the Special Issue Robust Procedures for Estimating and Testing in the Framework of Divergence Measures)

► Show Figures

Graphical abstract

17 pages, 354 KiB

Open AccessArticle

Monitoring Parameter Change for Time Series Models of Counts Based on Minimum Density Power Divergence Estimator

by Sangyeol Lee and Dongwon Kim

Entropy 2020, 22(11), 1304; https://doi.org/10.3390/e22111304 - 16 Nov 2020

Cited by 6 | Viewed by 2414

Abstract

In this study, we consider an online monitoring procedure to detect a parameter change for integer-valued generalized autoregressive heteroscedastic (INGARCH) models whose conditional density of present observations over past information follows one parameter exponential family distributions. For this purpose, we use the cumulative [...] Read more.

In this study, we consider an online monitoring procedure to detect a parameter change for integer-valued generalized autoregressive heteroscedastic (INGARCH) models whose conditional density of present observations over past information follows one parameter exponential family distributions. For this purpose, we use the cumulative sum (CUSUM) of score functions deduced from the objective functions, constructed for the minimum power divergence estimator (MDPDE) that includes the maximum likelihood estimator (MLE), to diminish the influence of outliers. It is well-known that compared to the MLE, the MDPDE is robust against outliers with little loss of efficiency. This robustness property is properly inherited by the proposed monitoring procedure. A simulation study and real data analysis are conducted to affirm the validity of our method. Full article

(This article belongs to the Special Issue Robust Procedures for Estimating and Testing in the Framework of Divergence Measures)

► Show Figures

Figure 1

121 pages, 1378 KiB

Open AccessArticle

by Niels B. Kammerer and Wolfgang Stummer

Entropy 2020, 22(8), 874; https://doi.org/10.3390/e22080874 - 8 Aug 2020

Cited by 3 | Viewed by 3217

Abstract

We compute exact values respectively bounds of dissimilarity/distinguishability measures–in the sense of the Kullback-Leibler information distance (relative entropy) and some transforms of more general power divergences and Renyi divergences–between two competing discrete-time Galton-Watson branching processes with immigration GWI for which the offspring as [...] Read more.

We compute exact values respectively bounds of dissimilarity/distinguishability measures–in the sense of the Kullback-Leibler information distance (relative entropy) and some transforms of more general power divergences and Renyi divergences–between two competing discrete-time Galton-Watson branching processes with immigration GWI for which the offspring as well as the immigration (importation) is arbitrarily Poisson-distributed; especially, we allow for arbitrary type of extinction-concerning criticality and thus for non-stationarity. We apply this to optimal decision making in the context of the spread of potentially pandemic infectious diseases (such as e.g., the current COVID-19 pandemic), e.g., covering different levels of dangerousness and different kinds of intervention/mitigation strategies. Asymptotic distinguishability behaviour and diffusion limits are investigated, too. Full article

(This article belongs to the Special Issue Robust Procedures for Estimating and Testing in the Framework of Divergence Measures)

► Show Figures

Figure 1

22 pages, 392 KiB

Open AccessArticle

Robust Change Point Test for General Integer-Valued Time Series Models Based on Density Power Divergence

by Byungsoo Kim and Sangyeol Lee

Entropy 2020, 22(4), 493; https://doi.org/10.3390/e22040493 - 24 Apr 2020

Cited by 12 | Viewed by 3395

Abstract

In this study, we consider the problem of testing for a parameter change in general integer-valued time series models whose conditional distribution belongs to the one-parameter exponential family when the data are contaminated by outliers. In particular, we use a robust change point [...] Read more.

In this study, we consider the problem of testing for a parameter change in general integer-valued time series models whose conditional distribution belongs to the one-parameter exponential family when the data are contaminated by outliers. In particular, we use a robust change point test based on density power divergence (DPD) as the objective function of the minimum density power divergence estimator (MDPDE). The results show that under regularity conditions, the limiting null distribution of the DPD-based test is a function of a Brownian bridge. Monte Carlo simulations are conducted to evaluate the performance of the proposed test and show that the test inherits the robust properties of the MDPDE and DPD. Lastly, we demonstrate the proposed test using a real data analysis of the return times of extreme events related to Goldman Sachs Group stock. Full article

(This article belongs to the Special Issue Robust Procedures for Estimating and Testing in the Framework of Divergence Measures)

► Show Figures

Figure 1

17 pages, 778 KiB

Open AccessArticle

Robust Regression with Density Power Divergence: Theory, Comparisons, and Data Analysis

by Marco Riani, Anthony C. Atkinson, Aldo Corbellini and Domenico Perrotta

Entropy 2020, 22(4), 399; https://doi.org/10.3390/e22040399 - 31 Mar 2020

Cited by 13 | Viewed by 4820

Abstract

Minimum density power divergence estimation provides a general framework for robust statistics, depending on a parameter

α

, which determines the robustness properties of the method. The usual estimation method is numerical minimization of the power divergence. The paper considers the special case [...] Read more.

Minimum density power divergence estimation provides a general framework for robust statistics, depending on a parameter

α

, which determines the robustness properties of the method. The usual estimation method is numerical minimization of the power divergence. The paper considers the special case of linear regression. We developed an alternative estimation procedure using the methods of S-estimation. The rho function so obtained is proportional to one minus a suitably scaled normal density raised to the power

α

. We used the theory of S-estimation to determine the asymptotic efficiency and breakdown point for this new form of S-estimation. Two sets of comparisons were made. In one, S power divergence is compared with other S-estimators using four distinct rho functions. Plots of efficiency against breakdown point show that the properties of S power divergence are close to those of Tukey’s biweight. The second set of comparisons is between S power divergence estimation and numerical minimization. Monitoring these two procedures in terms of breakdown point shows that the numerical minimization yields a procedure with larger robust residuals and a lower empirical breakdown point, thus providing an estimate of

α

leading to more efficient parameter estimates. Full article

(This article belongs to the Special Issue Robust Procedures for Estimating and Testing in the Framework of Divergence Measures)

► Show Figures

Figure 1

24 pages, 3694 KiB

Open AccessArticle

Robust Model Selection Criteria Based on Pseudodistances

by Aida Toma, Alex Karagrigoriou and Paschalini Trentou

Entropy 2020, 22(3), 304; https://doi.org/10.3390/e22030304 - 6 Mar 2020

Cited by 18 | Viewed by 3086

Abstract

In this paper, we introduce a new class of robust model selection criteria. These criteria are defined by estimators of the expected overall discrepancy using pseudodistances and the minimum pseudodistance principle. Theoretical properties of these criteria are proved, namely asymptotic unbiasedness, robustness, consistency, [...] Read more.

In this paper, we introduce a new class of robust model selection criteria. These criteria are defined by estimators of the expected overall discrepancy using pseudodistances and the minimum pseudodistance principle. Theoretical properties of these criteria are proved, namely asymptotic unbiasedness, robustness, consistency, as well as the limit laws. The case of the linear regression models is studied and a specific pseudodistance based criterion is proposed. Monte Carlo simulations and applications for real data are presented in order to exemplify the performance of the new methodology. These examples show that the new selection criterion for regression models is a good competitor of some well known criteria and may have superior performance, especially in the case of small and contaminated samples. Full article

(This article belongs to the Special Issue Robust Procedures for Estimating and Testing in the Framework of Divergence Measures)

► Show Figures

Figure 1

20 pages, 352 KiB

Open AccessArticle

Model Selection in a Composite Likelihood Framework Based on Density Power Divergence

by Elena Castilla, Nirian Martín, Leandro Pardo and Konstantinos Zografos

Entropy 2020, 22(3), 270; https://doi.org/10.3390/e22030270 - 27 Feb 2020

Cited by 4 | Viewed by 3934

Abstract

This paper presents a model selection criterion in a composite likelihood framework based on density power divergence measures and in the composite minimum density power divergence estimators, which depends on an tuning parameter

α

. After introducing such a criterion, some asymptotic properties [...] Read more.

This paper presents a model selection criterion in a composite likelihood framework based on density power divergence measures and in the composite minimum density power divergence estimators, which depends on an tuning parameter

α

. After introducing such a criterion, some asymptotic properties are established. We present a simulation study and two numerical examples in order to point out the robustness properties of the introduced model selection criterion. Full article

(This article belongs to the Special Issue Robust Procedures for Estimating and Testing in the Framework of Divergence Measures)

42 pages, 719 KiB

Open AccessArticle

Convergence Rates for Empirical Estimation of Binary Classification Bounds

by Salimeh Yasaei Sekeh, Morteza Noshad, Kevin R. Moon and Alfred O. Hero

Entropy 2019, 21(12), 1144; https://doi.org/10.3390/e21121144 - 23 Nov 2019

Cited by 6 | Viewed by 3370

Abstract

Bounding the best achievable error probability for binary classification problems is relevant to many applications including machine learning, signal processing, and information theory. Many bounds on the Bayes binary classification error rate depend on information divergences between the pair of class distributions. Recently, [...] Read more.

Bounding the best achievable error probability for binary classification problems is relevant to many applications including machine learning, signal processing, and information theory. Many bounds on the Bayes binary classification error rate depend on information divergences between the pair of class distributions. Recently, the Henze–Penrose (HP) divergence has been proposed for bounding classification error probability. We consider the problem of empirically estimating the HP-divergence from random samples. We derive a bound on the convergence rate for the Friedman–Rafsky (FR) estimator of the HP-divergence, which is related to a multivariate runs statistic for testing between two distributions. The FR estimator is derived from a multicolored Euclidean minimal spanning tree (MST) that spans the merged samples. We obtain a concentration inequality for the Friedman–Rafsky estimator of the Henze–Penrose divergence. We validate our results experimentally and illustrate their application to real datasets. Full article

(This article belongs to the Special Issue Robust Procedures for Estimating and Testing in the Framework of Divergence Measures)

► Show Figures

Figure 1

Journal Menu

Journal Browser

Robust Procedures for Estimating and Testing in the Framework of Divergence Measures

Share This Special Issue

Special Issue Editors

Special Issue Information

Keywords

Benefits of Publishing in a Special Issue

Related Special Issue

Published Papers (10 papers)

Editorial

Research

Further Information

Guidelines

MDPI Initiatives

Follow MDPI