Complexity, Criticality and Computation

What makes a system “complex”? Is it merely the number of components it integrates, a nonlinear nature of the dependencies and feedbacks among its parts, or an unpredictable behavior it exhibits over time? The term “complexity” was initially applied generically to express the lack of predictability, reflecting on the self-organization of a synergistic macroscopic behavior out of interactions between the constituent microscopic parts, and the emergence of global patterns. Without a doubt, by now the concept has acquired a fairly definitive meaning, describing a distinct field of research and education and a new approach to science and engineering. There are abundant examples showing that the enterprise of Complex Systems, having achieved a substantial level of maturity, reaches back into our everyday lives, revealing patterns of complexity that should be considered without employing a reductionist logic [1].

Similarly, the idea of criticality was originally motivated by studies of various crises and disruptive events, as well as sensitivities to initial conditions, but over time has developed into a precise field: critical dynamics. Research into critical dynamics is typically focused on the behavior of dynamical spatiotemporal systems during phase transitions where scale invariance prevails and symmetries break. Crucially, such behavior can be understood in terms of the control and order parameters. For instance, a second-order phase transition in a ferromagnetic system, separating two qualitatively different behaviors, can be reached by controlling the temperature parameter: the “disordered” and isotropic (symmetric) high-temperature phase is characterized by the absence of net magnetization, while the “ordered” and anisotropic (less symmetric) low-temperature phase can be described by an order parameter, the net magnetization vector defining a preferred direction in space. Critical phenomena have become associated with the physics of critical points, such as fractal behavior, the divergence of the correlation length, power-law divergences (e.g., the divergence of the magnetic susceptibility in the ferromagnetic phase transition), universality of relevant critical exponents, and so on. Now, these precise theoretical notions begin to reconnect with their motivating applied studies of crisis modeling, forecasting, and response. There is a growing awareness that complexity is strongly related to criticality, and many examples of self-organizing complex systems can be found in applications managing complexity specifically at critical regimes.

A similar loop originating in practical studies, maturing to an exact science with precise but narrow definitions, and then reaching back to applied scenarios, can be seen in the realm of distributed computation. These days, complex systems can be viewed as distributed information-processing systems, in the domains ranging from systems biology and artificial life to computational neuroscience, digital circuitry, and transport networks [2]. Consciousness emerging from neuronal activity and interactions, cell behavior resultant from gene regulatory networks, and flocking and swarming behaviors are all examples of global system behavior emerging as a result of the local interactions of the individuals (neurons, genes, animals). Can these interactions be seen as a generic computational process? This question shapes the third component of our Special Issue, linking computation to complexity and criticality.

The issue begins with three papers which deal with the foundational aspects of information processing in complex systems [3,4,5]. The study of Allen et al. [3] describes two quantitative indices that summarize the structure of a complex system: (i) its complexity profile, based on the multivariate mutual information at a given scale or higher, and (ii) the marginal utility of information, characterizing the extent to which a system can be described using limited amounts of information. Information is understood to have a scale equal to the multiplicity (or redundancy) at which it arises, and so the analysis shows how these indices capture the multi-scale structure of complex systems. The work of Chicharro and Panzeri [4] also deals with the redundant aspects of information: it extends the framework of mutual information decomposition, based on the construction of information gain lattices, separating the information into the unique, redundant, and synergy components. In doing so, the work proposes a new construction of information gain and loss lattices. The framework developed by Biehl et al. [5] presents a novel formal analysis of the specific and complete local integration of entities within distributed dynamical systems (e.g., Bayesian networks), and puts it in the context of measures of complexity and information integration, as well as multi-information. The analysis presented in this paper goes to the core of complexity phenomenon, seen through the lens of synergistic integrative organization, viewing entities as patterns occurring within a spatiotemporal trajectory.

A cross-disciplinary connection between information-theoretic and game-theoretic aspects of complexity and computation is explored in the study of Harré [6], which focuses on the mutual information between previous game states and an agent’s next action. This reveals a novel connection between the computational principles of logic gates, the structure of games, and the agents’ decision strategies.

Nonlinear dynamics and inherent feedbacks typical in complex systems are considered in the next investigation by Zhang et al. [7], which addresses the problem of achieving and maintaining consensus in second-order multi-agent systems. This problem is pertinent to several scenarios, such as distributed control in networks of mass-spring systems, synchronization of coupled harmonic oscillators, and stability analysis of power systems. The study produces an adaptive consensus protocol for the problem’s variant with an exogenous disturbance generated by an unknown exogenous system.

The next four papers [8,9,10,11] are placed in the complexity–criticality–computation overlap which is central to our issue. The study of Erten et al. [8] continues the information-theoretic theme by applying the information dynamics framework to studies of critical thresholds during epidemics. The approach uses the transfer entropy as a measure of distributed communications during a network-wide contagion seen as computation, as well as the active information storage as a measure of the corresponding distributed memory. The results for finite-size systems identify a critical interval, rather than an exact critical threshold. The methods described by Roli et al. [9] also detect criticality; that is, they distinguish between different phases separated by a critical regime. The approach is centered on the relevance index—an information-theoretic ratio relating the multi-information (or integration) measure to the mutual information between a subsystem and the rest of the system. The reported results demonstrate that the relevance index is consistently maximized at the critical regime. A phase transition-like behavior is investigated in the paper by Kramer et al. [10] as well. Their work identifies qualitative changes such as macroscopic spatiotemporal pattern formation in dynamics of Cellular Automata, by varying the inertia—an inner resistance to changes within cells—as the control parameter. In an ecological context, the inertia is related to an impairment and competition between species. The study by Mayer [11] illustrates the effects of critical connectivity in echo state networks and identifies under which conditions the recurrent connectivity is achieved. The results are contrasted with alternative approaches considering the dynamics near the “edge of chaos”. The overall approach opens a way to organize reservoirs of neuronal connections as recurrent filters with a memory compression feature.

The three final studies are also biologically motivated. Continuing with the topic of neuronal connectivity, the paper by Kunert-Graf [12] attempts to identify the source of complexity in the biological neuronal network of C. elegans, the only organism for which its “connectome” is known. Using a suitably defined measure (once again based on information theory), the study argues that the somatic nervous system of C. elegans is much more complex than a random graph with the same degree distribution. The complexity and efficiency of solutions evolved by nature is a source of inspiration for another study, in which Kwiecień and Pasieka [13] use a computational swarm optimization algorithm to solve a travel planning problem. The presented approach is found to outperform the particle swarm optimization algorithm. The analysis presented by Farnsworth [14] brings the subject of distributed information processing to “the far end of the complexity gradient”, centering the discussion on the question of free-will in artificial agents. Not surprisingly, this thought-provoking examination highlights the role of information in shaping the interactions and dynamics among patterns, as well as the distribution of matter and energy in space and time. The work concludes with the conjecture that free-will—which currently remains a property of living things—may still be attained in synthetic robots.

The contributions to this special issue show that the overlap between complexity, criticality, and computation provides fertile ground with both theoretical and practical dimensions. Considering complex systems as dynamical systems performing distributed computation suggests a unifying perspective, which reveals key thermodynamic and information-processing components, as well as their behavior near critical regimes. These components (e.g., collective memory, long-range communications, and synergistic modifications), together with the consequent physical fluxes [15], can be quantified and optimized. In spirit of Guided Self-Organization [2], the resultant dynamics can then be guided towards desired regions of the corresponding state-spaces, combining the power and efficiency of the self-organization so abundant in nature with the accuracy and reliability of traditional design approaches.