Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
4.9
2.1
Journals
Entropy
Special Issues
Information-Theoretic Approaches to Explaining Linguistic Structure
entropy-logo
Submit to Entropy Review for Entropy Propose a Special Issue

Journal Menu

► Journal Menu

Journal Browser

► Journal Browser
share announcement

Information-Theoretic Approaches to Explaining Linguistic Structure

A special issue of Entropy (ISSN 1099-4300). This special issue belongs to the section "Information Theory, Probability and Statistics".

Deadline for manuscript submissions: closed (15 December 2021) | Viewed by 20607

Special Issue Editors

Dr. Kathleen Currie Hall

E-Mail Website
Guest Editor
Department of Linguistics, University of British Columbia, Vancouver, BC V6T 1Z4, Canada
Interests: laboratory phonology; information theory; corpus phonology; signed languages
Dr. Uriel Cohen Priva

E-Mail Website
Guest Editor
Department of Cognitive, Linguistic & Psychological Sciences, Brown University, Providence, RI 02912, USA
Interests: phonology; functional pressures on language; information theory and language
Dr. Richard Futrell

E-Mail Website
Guest Editor
Department of Language Science, University of California Irvine, Irvine, CA 92697, USA
Interests: language processing; typology; syntax

Special Issue Information

Dear Colleagues,

Information theory is a highly generic and powerful mathematical framework for analyzing communication systems. In recent years, there has been renewed interest in using this framework to understand linguistic structures. This is in part because language is a communication system that enables effective communication, subject to cognitive, physical, and social constraints, on how we encode, transmit, receive, decode, and store linguistic content. Information theory provides ways not only to formalize these constraints, but also ways to study how they affect the structure of the resulting communication systems. Information theory thus provides a bridge between linguistic function and linguistic form.

In this Special Issue, we invite contributions applying information theory to explain why and how particular linguistic phenomena arise at all levels of linguistic analysis, such as phonetics, phonology, morphology, syntax, semantics, and pragmatics, as well as cross-cutting areas such as sociolinguistics, historical linguistics, acquisition, and language processing.

We hope this collection of papers will serve to create a theoretical common ground both within linguistics, and between linguistics and other fields that use information-theoretic principles as explanatory tools.

 

Dr. Kathleen Currie Hall
Dr. Uriel Cohen Priva
Dr. Richard Futrell
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Entropy is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • information theory
  • linguistic structure
  • linguistic explanation
  • phonetics
  • phonology
  • morphology
  • syntax
  • semantics
  • pragmatics
  • historical linguistics
  • sociolinguistics
  • language processing
  • language acquisition

Benefits of Publishing in a Special Issue

  • Ease of navigation: Grouping papers by topic helps scholars navigate broad scope journals more efficiently.
  • Greater discoverability: Special Issues support the reach and impact of scientific research. Articles in Special Issues are more discoverable and cited more frequently.
  • Expansion of research network: Special Issues facilitate connections among authors, fostering scientific collaborations.
  • External promotion: Articles in Special Issues are often promoted through the journal's social media, increasing their visibility.
  • e-Book format: Special Issues with more than 10 articles can be published as dedicated e-books, ensuring wide and rapid dissemination.

Further information on MDPI's Special Issue polices can be found here.

Published Papers (6 papers)

Download All Papers
Order results
Result details
Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:

Research

15 pages, 7156 KiB  
Article
Grammatical Gender Disambiguates Syntactically Similar Nouns
by Phillip G. Rogers and Stefan Th. Gries
Entropy 2022, 24(4), 520; https://doi.org/10.3390/e24040520 - 7 Apr 2022
Cited by 4 | Viewed by 2944
Abstract
Recent research into grammatical gender from the perspective of information theory has shown how seemingly arbitrary gender systems can ease processing demands by guiding lexical prediction. When the gender of a noun is revealed in a preceding element, the list of possible candidates [...] Read more.
Recent research into grammatical gender from the perspective of information theory has shown how seemingly arbitrary gender systems can ease processing demands by guiding lexical prediction. When the gender of a noun is revealed in a preceding element, the list of possible candidates is reduced to the nouns assigned to that gender. This strategy can be particularly effective if it eliminates words that are likely to compete for activation against the intended word. We propose syntax as the crucial context within which words must be disambiguated, hypothesizing that syntactically similar words should be less likely to share a gender cross-linguistically. We draw on recent work on syntactic information in the lexicon to define the syntactic distribution of a word as a probability vector of its participation in various dependency relations, and we extract such relations for 32 languages from the Universal Dependencies Treebanks. Correlational and mixed-effects regression analyses reveal that syntactically similar nouns are less likely to share a gender, the opposite pattern that is found for semantically and orthographically similar words. We interpret this finding as a design feature of language, and this study adds to a growing body of research attesting to the ways in which functional pressures on learning, memory, production, and perception shape the lexicon in different ways. Full article
(This article belongs to the Special Issue Information-Theoretic Approaches to Explaining Linguistic Structure)
►▼ Show Figures

Figure 1

17 pages, 343 KiB  
Article
Evolution and Trade-Off Dynamics of Functional Load
by Erich Round, Rikker Dockum and Robin J. Ryder
Entropy 2022, 24(4), 507; https://doi.org/10.3390/e24040507 - 5 Apr 2022
Cited by 1 | Viewed by 2709
Abstract
Functional load (FL) quantifies the contributions by phonological contrasts to distinctions made across the lexicon. Previous research has linked particularly low values of FL to sound change. Here, we broaden the scope of enquiry into FL to its evolution at higher values also. [...] Read more.
Functional load (FL) quantifies the contributions by phonological contrasts to distinctions made across the lexicon. Previous research has linked particularly low values of FL to sound change. Here, we broaden the scope of enquiry into FL to its evolution at higher values also. We apply phylogenetic methods to examine the diachronic evolution of FL across 90 languages of the Pama–Nyungan (PN) family of Australia. We find a high degree of phylogenetic signal in FL, indicating that FL values covary closely with genealogical structure across the family. Though phylogenetic signals have been reported for phonological structures, such as phonotactics, their detection in measures of phonological function is novel. We also find a significant, negative correlation between the FL of vowel length and of the following consonant—that is, a time-depth historical trade-off dynamic, which we relate to known allophony in modern PN languages and compensatory sound changes in their past. The findings reveal a historical dynamic, similar to transphonologization, which we characterize as a flow of contrastiveness between subsystems of the phonology. Recurring across a language family that spans a whole continent and many millennia of time depth, our findings provide one of the most compelling examples yet of Sapir’s ‘drift’ hypothesis of non-accidental parallel development in historically related languages. Full article
(This article belongs to the Special Issue Information-Theoretic Approaches to Explaining Linguistic Structure)
►▼ Show Figures

Figure 1

16 pages, 1094 KiB  
Article
Frequency, Informativity and Word Length: Insights from Typologically Diverse Corpora
by Natalia Levshina
Entropy 2022, 24(2), 280; https://doi.org/10.3390/e24020280 - 16 Feb 2022
Cited by 9 | Viewed by 4004
Abstract
Zipf’s law of abbreviation, which posits a negative correlation between word frequency and length, is one of the most famous and robust cross-linguistic generalizations. At the same time, it has been shown that contextual informativity (average surprisal given previous context) is more strongly [...] Read more.
Zipf’s law of abbreviation, which posits a negative correlation between word frequency and length, is one of the most famous and robust cross-linguistic generalizations. At the same time, it has been shown that contextual informativity (average surprisal given previous context) is more strongly correlated with word length, although this tendency is not observed consistently, depending on several methodological choices. The present study examines a more diverse sample of languages than the previous studies (Arabic, Finnish, Hungarian, Indonesian, Russian, Spanish and Turkish). I use large web-based corpora from the Leipzig Corpora Collection to estimate word lengths in UTF-8 characters and in phonemes (for some of the languages), as well as word frequency, informativity given previous word and informativity given next word, applying different methods of bigrams processing. The results show different correlations between word length and the corpus-based measure for different languages. I argue that these differences can be explained by the properties of noun phrases in a language, most importantly, by the order of heads and modifiers and their relative morphological complexity, as well as by orthographic conventions. Full article
(This article belongs to the Special Issue Information-Theoretic Approaches to Explaining Linguistic Structure)
►▼ Show Figures

Figure 1

21 pages, 1894 KiB  
Article
Quantifiers in Natural Language: Efficient Communication and Degrees of Semantic Universals
by Shane Steinert-Threlkeld
Entropy 2021, 23(10), 1335; https://doi.org/10.3390/e23101335 - 14 Oct 2021
Cited by 12 | Viewed by 3439
Abstract
While the languages of the world vary greatly, they exhibit systematic patterns, as well. Semantic universals are restrictions on the variation in meaning exhibit cross-linguistically (e.g., that, in all languages, expressions of a certain type can only denote meanings with a certain special [...] Read more.
While the languages of the world vary greatly, they exhibit systematic patterns, as well. Semantic universals are restrictions on the variation in meaning exhibit cross-linguistically (e.g., that, in all languages, expressions of a certain type can only denote meanings with a certain special property). This paper pursues an efficient communication analysis to explain the presence of semantic universals in a domain of function words: quantifiers. Two experiments measure how well languages do in optimally trading off between competing pressures of simplicity and informativeness. First, we show that artificial languages which more closely resemble natural languages are more optimal. Then, we introduce information-theoretic measures of degrees of semantic universals and show that these are not correlated with optimality in a random sample of artificial languages. These results suggest both that efficient communication shapes semantic typology in both content and function word domains, as well as that semantic universals may not stand in need of independent explanation. Full article
(This article belongs to the Special Issue Information-Theoretic Approaches to Explaining Linguistic Structure)
►▼ Show Figures

Figure 1

14 pages, 340 KiB  
Article
Characterizing the Typical Information Curves of Diverse Languages
by Josef Klafka and Daniel Yurovsky
Entropy 2021, 23(10), 1300; https://doi.org/10.3390/e23101300 - 2 Oct 2021
Cited by 2 | Viewed by 2287
Abstract
Optimal coding theories of language predict that speakers will keep the amount of information in their utterances relatively uniform under the constraints imposed by their language, but how much do these constraints influence information structure, and how does this influence vary across languages? [...] Read more.
Optimal coding theories of language predict that speakers will keep the amount of information in their utterances relatively uniform under the constraints imposed by their language, but how much do these constraints influence information structure, and how does this influence vary across languages? We present a novel method for characterizing the information structure of sentences across a diverse set of languages. While the structure of English is broadly consistent with the shape predicted by optimal coding, many languages are not consistent with this prediction. We proceed to show that the characteristic information curves of languages are partly related to a variety of typological features from phonology to word order. These results present an important step in the direction of exploring upper bounds for the extent to which linguistic codes can be optimal for communication. Full article
(This article belongs to the Special Issue Information-Theoretic Approaches to Explaining Linguistic Structure)
►▼ Show Figures

Figure 1

35 pages, 507 KiB  
Article
A Refutation of Finite-State Language Models through Zipf’s Law for Factual Knowledge
by Łukasz Dębowski
Entropy 2021, 23(9), 1148; https://doi.org/10.3390/e23091148 - 1 Sep 2021
Cited by 3 | Viewed by 3185
Abstract
We present a hypothetical argument against finite-state processes in statistical language modeling that is based on semantics rather than syntax. In this theoretical model, we suppose that the semantic properties of texts in a natural language could be approximately captured by a recently [...] Read more.
We present a hypothetical argument against finite-state processes in statistical language modeling that is based on semantics rather than syntax. In this theoretical model, we suppose that the semantic properties of texts in a natural language could be approximately captured by a recently introduced concept of a perigraphic process. Perigraphic processes are a class of stochastic processes that satisfy a Zipf-law accumulation of a subset of factual knowledge, which is time-independent, compressed, and effectively inferrable from the process. We show that the classes of finite-state processes and of perigraphic processes are disjoint, and we present a new simple example of perigraphic processes over a finite alphabet called Oracle processes. The disjointness result makes use of the Hilberg condition, i.e., the almost sure power-law growth of algorithmic mutual information. Using a strongly consistent estimator of the number of hidden states, we show that finite-state processes do not satisfy the Hilberg condition whereas Oracle processes satisfy the Hilberg condition via the data-processing inequality. We discuss the relevance of these mathematical results for theoretical and computational linguistics. Full article
(This article belongs to the Special Issue Information-Theoretic Approaches to Explaining Linguistic Structure)
Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:
 
Displaying articles 1-6
Back to TopTop