From Hearing to Listening: Design and Properties of an Actively Tunable Electronic Hearing Sensor

Abstract

An important step towards understanding the working principles of the mammalian hearing sensor is the concept of an active cochlear amplifier. Theoretical arguments and physiological measurements suggest that the active cochlear amplifiers originate from systems close to a Hopf bifurcation. Efforts to model the mammalian hearing sensor on these grounds have, however, either had problems in reproducing sufficiently close essential aspects of the biological example (Magnasco, M.O. Phys. Rev. Lett. 90, 058101 (2003); Duke, T. & Jülicher, F. Phys. Rev. Lett. 90, 158101 (2003)), or required complicated spatially coupled differential equation systems that are unfeasible for transient signals (Kern, A. & Stoop, R. Phys. Rev. Lett. 91, 128101 (2003)). Here, we demonstrate a simple system of electronically coupled Hopf amplifiers that not only leads to the desired biological response behavior, but also has real-time capacity. The obtained electronic Hopf cochlea shares all salient signal processing features exhibited by the mammalian cochlea and thus provides a simple and efficient design of an artificial mammalian hearing sensor.

1. Introduction

Understanding hearing in general and more specifically the mammalian hearing organ, the cochlea, is a long-standing human endeavour. The first basic step was taken by H.L.F. Helmholtz, who proposed in 1863 [1] the existence of a one-to-one correspondence between a position along the cochlear duct and a stimulation frequency to which, at the given position, the response is maximal (the tonotopic principle). Important steps that followed were von Békésy's discovery of traveling waves along the basilar membrane (BM) as the carriers of the auditory information (1928 [2]) and Gold's conjecture of an active amplification process within the cochlea (1948 [3]). The latter phenomenon was corroborated by the discovery of otoacoustic emissions (1978 [4]), the autonomous generation of sounds by the cochlea itself. Subsequent experiments have revealed that the active amplification is located in the outer hair cells (OHC) attached to the basilar membrane [5, 6]. It has been known for some time that systems close to bifurcation instabilities can act as small-signal amplifiers [7, 8], where each bifurcation type ads its own amplification signature.

Physiological measurements have shown that among mammals, the hearing characteristics vary little [9] and the measurement details [10] imply that the instability responsible for active amplification is of Hopf type. This led Eguíluz et al. [11] to propose the use of Hopf-type amplifiers as the basic elements for cochlear modeling (for a related result, see [12]). They argued that the Hopf nonlinearities could correctly capture the basic aspects of mammalian hearing: compression of the dynamic range; sharper tuning for lower intensity sounds, and the generation of combination tones. In turtles and bullfrogs, cells that are homologous to the mammalian hair cells (see [13]) have already been shown to display amplification profiles that are characteristic for systems close to a Hopf instability [14].

In the cochlea, pressure variations generated by incoming sounds are transformed into incompressible and inviscid hydrodynamic waves. As these waves move down the cochlea, they cause small BM displacements [15]. Using x to denote the distance from the stapes along the unrolled cochlea, the system can be linearly described by a water-surface wave with fluid depth h, density ρ, surface mass density m and exponentially decreasing transversal stiffness [16]. In this description, using a dispersion relation, the differential equation ruling the spatial energy distribution e(x) obtains the form [17]

$$\frac{\partial e}{\partial x}=\frac{-1}{v_G(x,\omega )}\left[\frac{\partial v_G(x,\omega )}{\partial x}+d(x,\omega )\right]e+\frac{a(x,e,\omega )}{v_G(x,\omega )},$$

(1)

where a(·) the power supplied by the local active amplification which works against the internal viscous losses d(x, ω) = 4ν k(x, ω)², ν the kinematic viscosity. v_G is the group velocity and k the wave number of the stimulating wave [18, 19]. Following the biological example, the active amplification results from an array of Hopf-type power sources aligned along the BM, where each amplifier has its own natural (or: ‘characteristic’) frequency ω_ch(x). Given a forcing frequency ω, the Hopf amplifiers with ω_ch(x) ≈ ω are maximally excited at locations x_ch(ω) < x_c(ω), beyond which viscosity leads to a precipitous decay of the wave amplitude.

The characteristics of the active contribution a are derived from a ω_ch-rescaled Hopf differential equation

$$\dot{\mathbf{z}}=(\mu +j){\omega }_{ch}\mathbf{z}-w_{ch}{\left|\mathbf{z}\right|}^2\mathbf{z}-{\omega }_{ch}\mathbf{F}(t),\mathbf{z}\in ℂ,$$

(2)

where j is the imaginary unit. Assuming a 1:1 locking between signal and system, z(t) = Re^j(ωt+θ) is the amplified external periodic input F = Fe^jωt, ω_ch is the natural frequency of the oscillation, and μ ∈ ℝ denotes the Hopf nonlinearity parameter. For F = 0, Eq. (2) describes the generic differential equation displaying a Hopf bifurcation: For μ < 0, the solution z(t) = 0 is a stable fixed point, whereas for μ > 0, the fixed-point solution becomes unstable and a stable limit-cycle of the form $\mathbf{\text{z}}(t)=\sqrt{\mu }{e}^{j{\omega }_{ch}t}$ appears. For time-varying signals F(t) it is convenient to put a handle on the response latency, by multiplying the damping term ω_ch |z|² z by a factor γ > 0. A nonzero forcing F then yields ω_chFe^−jθ = (μ + j) ω_chR−γω_chR³−jωR. Evaluation of the squared modulus and introducing the variable $\varphi =\frac{\omega }{{\omega }_{ch}}$ results in

$$F^2={\gamma }^2{R}^6-2\gamma \mu R^4+\left[{\mu }^2+{(1-\varphi )}^2\right]R^2,$$

(3)

which is easily solved. For μ = 0 and close to resonance (ω = ω_ch), the response R = F^1/3 emerges, which forces the gain G = R/F = F^−2/3 to increase towards infinity as F approaches zero. For μ < 0, maintaining ω = ω_ch, we obtain the response R = −F/μ for weak stimuli F. As F increases, the term R⁶ in Eq. (3) can no longer be neglected, and the compressive nonlinear regime is entered (γR⁶ ≈ μ²R² + 2γμR⁴). The transition point is located at $F_C(\mu )\approx 0.91{(-\mu )}^{3/2}/{\gamma }^{\frac{1}{2}}$. Therefore, for weak stimuli F, the response R is almost linear; for moderate stimuli the differential gain of the system, dR/dF, decreases with increasing stimulus intensity. As we move away from resonance, the last term of Eq. (3) dominates, leading, as R ≈ F/|1 − ϕ|, to a linear response, irrespective of the stimulation strength. For μ > 0, stable limit-cycles emerge. For an intuitive understanding, at a nonzero stimulation, the Hopf equation (2) can be interpreted as a nonlinear filter characterized by a tunable gain control (“quality factor”) |μ| and an envelope detector |z|². As the bandwidth Γ ∼ |μ| for F ≤ F_C (and Γ ∼ γ^1/2F^2/3 for F > F_C), small |μ|-values act as high Q-factors (sharp resonances).

2. Experimental setup

Electronic sensor design

For stationary inputs simulations of the differential equation (1) provide results that are close to those from the physiological example, if an appropriate coupling scheme is used [17]. In the simulation approaches pursued, the optimal coupling scheme however remained elusive to us. Only when using a simplification of the previously very detailed biologically motivated coupling schemes, our electronic implementation yielded results of the desired accuracy. The construction of the device now provides us with a handle to reconsider critically the biophysical basis used for the simulation approach, and to draw thereof corresponding biophysical conclusions.

In addition to providing suboptimal results, for transient signals our simulations were confronted with excessive computational demand. This was our main motivation to resort to an electronic implementation that, while being a simplification of the software model, should take care to preserve the insights gained. For the design, we decomposed the cochlea into n sections of characteristic frequencies ω_si, i = 1, … , n, and endowed each one with properties of the passive hydrodynamic behavior and an active Hopf amplifier. Major challenges were to properly connect the passive/active components towards a section, and the sections towards a cascade representing the entire cochlea. The Hopf part (2) can be approximated in circuitry by using a combination of integrative summers and multipliers, resistors R and a capitance C [20]. Using complex notation for the voltages v_z = v_x + jv_y, the Hopf equation reads

$${\dot{\mathbf{v}}}_{\mathbf{\text{z}}}=\left(\frac{-v_{\mu }}{mC{R}_{\mu }}+\frac{j}{C{R}_{\omega }}\right){\mathbf{v}}_{\mathbf{z}}-\frac{{\left|{\mathbf{v}}_{\mathbf{z}}\right|}^2{\mathbf{v}}_{\mathbf{z}}}{2m^{2}C{R}_{\gamma }}-\frac{{\mathbf{v}}_{\mathbf{F}}}{C{R}_F},$$

(4)

where V_F denotes the input voltage and v_μ, the control parameter. The multiplier m is implemented by means of analog multipliers ( $v_o=\frac{v_{i_0}\cdot v_{i_1}}{m}$ in electronic conventions). To map the Hopf system on the circuit, the interval of linear response was mapped on the non-saturation regime V_CC of the op-amp. We chose this interval as I_F = (0, F_C(μ*)], where μ* is a sufficiently negative value of μ where log[F_C] is essentially constant. For the proper mapping, we write v_z = A_zz, v_F = A_FF, v_μ = A_μμ. From equating (4) with (2), we obtain

$$\dot{\mathbf{z}}=\left(\frac{-A_{\mu }\mu }{mC{R}_{\mu }}+\frac{j}{C{R}_{\omega }}\right)\mathbf{z}-\frac{A_z^2{\left|\mathbf{z}\right|}^2\mathbf{z}}{2m^{2}C{R}_{\gamma }}-\frac{A_F\mathbf{F}}{C{R}_F{A}_z}.$$

(5)

The mapping thus requires to use A_F ≪ V_CC/F_C. We chose A_F = V_CC, and, to obtain a unitary gain, A_z = A_F. A_μ could be set arbitrarily, but is naturally chosen as A_μ = −V_CC/μ. In order to properly address situations where μ is changing over an interval I_μ, we replace μ by min(I_μ). Given capacitance C, the resistor values are calculated from the equivalence between Eq. (2) – modified by the factor γ – and Eq. (5) as

$$R_{\omega }=\frac{1}{C{\omega }_{ch}},R_{\mu }=\frac{-A_{\mu }}{mC{\omega }_{ch}},R_F=\frac{A_F}{A_{z}C{\omega }_{ch}},R_{\gamma }=\frac{A_z^2}{2m^{2}C\gamma {\omega }_{ch}}.$$

(6)

The responses generated by Eq. (1) [17] suggest that the passive part can be modeled as a 6^th-order Butterworth circuit. We chose to partition the 6^th-order Butterworth filter into three 2^nd-order low pass filters (see Fig. 1). Such a circuit transforms a rescaled input frequency $\widehat{\varphi }=\frac{\omega }{{\omega }_s}$ according to

$$\left|H_6^{{\omega }_s}(j\omega )\right|=:B_6(\widehat{\varphi })={\sum }_{i=0}^6\frac{1}{a_i{\widehat{\varphi }}^i}=\frac{1}{\left({\widehat{\varphi }}^2+0.518\widehat{\varphi }+1\right)}\cdot \frac{1}{\left({\widehat{\varphi }}^2+\sqrt{2}\widehat{\varphi }+1\right)}\cdot \frac{1}{\left({\widehat{\varphi }}^2+1.932\widehat{\varphi }+1\right)},$$

(7)

where $H_6^{{\omega }_s}(j\omega )$ is the transfer function of the 6^th-order filter with cutoff ω_s. The first filter part has a gain in excess of 1 at the characteristic frequency, leading to an op-amp saturation at large input voltages and small control values. This problem can be compensated for by changing the order of the 2^nd-order circuits. This does not compromise the amplification of small signals, if, following the biological example [17, 21], the Hopf amplifier preceeds the Butterworth filter. Putting the Hopf amplifier “in front” of the passive unit, and coupling the two units by means of a simplified feedforward coupling, avoids an uncontrollable interaction of the phases of the passive and the active components, a problem that feedback and open-loop gain amplifications usually struggle with. The corresponding section diagram is shown in Fig. 1. The cochlea was constructed by connecting sections of logarithmically increasing center frequencies ω_si in series. This implementation is different from the one used in our previous modeling studies [17]. Whereas in the latter the skeleton was provided by passive behavior, to which active components, coupled over the range of a few OHC, were added, in the present setting we have an enlarged range of coupling among combined active/passive units. The details of how this leads to the improved response by the hardware, seen from the biophysical measurements, and what biophysical consequences this has, will be the subject of a separate study.

In order to build a generic section, it was sufficient to specify the detuning between the passive frequency ω_si and the Hopf amplifier frequency, ϕ_s(i) := ω_si/ω_chi < 1, i = 1, … , n, where n is the number of sections. This was our first design parameter. The second design parameter was the relationship between the characteristic frequencies of subsequent section frequencies

$$\mathrm{\Psi }(i)=\frac{{\omega }_{s_{i+1}}}{{\omega }_{s_i}},i=1,\dots ,n-1.$$

(8)

For the realization of the cochlea reported below, we chose, for simplicity, the two parameters independent from the section, as Ψ(i) = Ψ, ϕ_s(i) = ϕ_s, ⩝i. For a frequency range to be covered, we used the second design parameter to determine the number of composing sections and to evaluate the characteristic frequencies. From Eqs. (6), the corresponding electronic components were obtained, which leaves as the only free parameter the Hopf parameter μ, in the electronic realization expressed by the gain control v_μ. As was outlined above, the Hopf parameter determines both the amplification strength and the tuning width (smaller |μ|'s lead to larger amplifications combined with sharper tuning widths). We chose an electronic realization that requires five sections to cover one octave. This choice provides an amplification profile close to that of the biophysical measurements, at a minimum number of components. The circuits were implemented using conventional hardware, obtainable in any electronic store. As a consequence, the supply voltage was chosen at 10 V. The electronic parts used were chosen according to cost and availability criteria only. A more careful design optimizing the impedances between the sections would substantially reduce the noise accumulated along the cochlea. Primary investigations have corroborated the expectation that the cochlea can easily be miniaturized and be cast in microelectronics. Complex sounds (such as the ‘click’ stimulations used) were obtained via a microphone (Sennheiser E845 with an appropriate pre-amplifier) or via the synthetic waveform generator (Agilent 33220A), followed by an analogue Hilbert transform that generated the real and the imaginary components of the signal. For the active tuning of the μ-parameter array we used the National Instruments 6723 I/O card.

3. Results

In what follows we report on a realization using design parameters ϕ_s = 1.05⁻¹ and Ψ = 0.84. This leads to a range of one octave (say the range from 1480 to 2960 Hz) by means of five components. The final cochlea prototype constructed embraced 24 sections.

Steady-state signal response

From the constructed electronic device, we measured the amplitude Amp = |v₀| generated in response to varying input frequencies and stimulation strengths (pure-tone stimulations). The comparison of the measurements after the first, the second, the tenth and the twentieth section demonstrates that by passing through the sections, the signal is gradually shaped (see Fig. 2), where after a few sections, the response already attains its characteristic form, see Fig. 3. Using more sections in the electronic device shapes the measured frequency response ever closer to the biophysical measurements (compare with [23], Fig. 1).

In Fig. 3, the response from four-section cochlea covering the range range from 5920 to 9956 Hz is taken for two values of the Hopf parameter μ, in order to demonstrate the influence μ has on the amplification strength. Close to bifurcation, a discretization effect is observed that limits the minimal number of sections that can be used given a required gain. The strength of the latter is a function of the frequency distance between adjacent sections (Ψ) and of the resonance widths of the active amplifiers (i.e., of μ). An array of about 30 sections leads to a hearing sensor that for practical purposes is comparable to the mammalian hearing organ.

Moreover, in experiments with stimulations by more than one tone, two-tone suppression and combination-tone generation, as the salient nonlinear phenomena of mammalian hearing [24–27], are reproduced with great fidelity, see Fig. 4.

Transient signal response

The response to transient, broadband stimulation is of similar quality. In Fig. 5, the results for click stimulations are displayed. A comparison with the corresponding biophysical measurements [28] reveals a close match. Also other biophysical experiments with transient signals reported in [28] are reproduced with ease. In Fig. 6, the instantaneous frequency response is displayed for different stimulus levels. The measured response reproduces the logarithmic increase towards the steady-state response near the central frequency (see inset), as well as the fact that the steady-state response is somewhat higher than the central frequency. Our measurements extend beyond the first wave packet, which leads at the packet interfaces to systematic outliers of the measured instantaneous frequencies. Even the phase functions taken from the device match with the ones taken from the biological example, see Fig. 7. Response phases increasingly lag as the frequency of stimulation increases. As in the biological example, a shallow slope at low frequencies is observed that turns into a steeper slope towards and beyond the central frequency.

Another transient signal of interest are low-frequency amplitude modulated sounds. That is because such signals play a significant role in clinical procedures for the differential diagnosis of sensory hearing loss. Moreover, such signals may play a major role in speech intelligibility. Also in this case, the correspondence between the physiological observations and the corresponding response generated by our electronic device is extremely close, see Fig. 8.

Active tuning towards signals

The least perfect correspondence between the signals delivered by our device and the physiological example is found in Fig. 5. In the electronic device, the response to sound stimulations is asymmetric (stronger in the beginning, lesser at the end of the signal), whereas in the biological example this is almost perfectly symmetrical. This discrepancy, however, can be fixed and understood by the following arguments. The physiological measurements indicate a response-dependent value of μ, potentially implemented on the level of hair cells with its channel dynamics (small |μ| at signal onset, larger values upon increased response). Before the arrival of a strong stimulating sound, the biological cochlea appears to be tuned for weaker sounds, using small values of the tuning parameter |μ|. With the arrival of a strong click, the amplification seems to be quickly decreased, and then kept constant. The observed behavior corresponds well to the behavior (latencies) one would expect if the hair-cells are tuned by means of its efferent connections, monitored by instructions from the auditory centers.

By closely following the biological example in our design, it is in the nature of the design of our cochlea that such a dynamical adaption of μ can be implemented with little additional effort. Even more importantly, on slower time-scales, the auditory neuronal feedback loop might tune μ in order to suppress unwanted, and to enhance desired, signal components. Theoretical experiments exploiting these features are successful and raise the expectation that by proceeding on this way, we may substantially reduce the cocktail-party problem. Preliminary results demonstrating this effect have already been obtained and corroborate the effectiveness of this feature.

4. Conclusions

We have demonstrated that the Hopf cochlear amplifier concept together with the cochlear biophysics thus provides the design of an extremely sensitive and robust hearing sensor, where salient nonlinear signal processing characteristics (compressive nonlinearity, high sensitivity, two-tone suppression, combination-tone generation) are naturally implemented. The device has all ingredients to serve as a perfect hearing replacement device (cochlear implant), once we succeed in similarly improving the stimulation electrodes needed for implants in a qualitatively comparable manner. Currently, such an electrode is being developed in collaboration with EMPA.