Randomness and Integral Forcing

1 Introduction

Many physical systems are governed by principles that can be expressed in terms of differential equations. In the case of a system with a multi-dimensional state vector x, the evolution of x is described by a set of differential equations, each taking the form:

x is a component of x, which is a function of time t. The differential forcing F(x(t)) is a deterministic function of x. F(x(t)) describes internal dynamics arising from interactions of x with other components of x under the influence of some external forcings. Examples of systems governed by equations in form of Eq.(1) include a climate model describing the atmosphere and the ocean, and a many-particle system describing the movements of Brownian particles suspended in a fluid. A common feature observed from these physical systems is the lack of serial correlations, where a solution at one time point is uncorrelated to the solution at another more distant time point. A solution that lacks serial correlation is commonly regarded as random. We identify this randomness as the subject of this study. Under this definition of randomness, movements of a Brownian particle are random; weather patterns are random. Random features are also found in many other occasions. A prominent example in atmospheric sciences concerns time averages of meteorological variables. These averages display variability similar to that of the sample mean of a random variable, leading to the concept known as “climate noise” (Leith, 1973; Madden, 1976, 1981; Feldstein & Robinson, 1994; Feldstein, 2000). Despite evident random behaviors found for classical physical systems, a theory of randomness is still missing.

Instead, heuristic arguments are used to provide some explanations. Such arguments often associate randomness with uncertainties. Two types of uncertainties are considered in this context. The first one arises from our inability to precisely track the evolution of each individual degree of freedom in a system that has an exceedingly large number of degrees of freedom. Brownian motion serves as a typical example, as it is challenging to formulate and to solve the complete set of equations that describe all interactions between fluid molecules and Brownian particles. The standard approach, commonly used to deal with noise and fluctuations in physical systems (MacDonald, 1962), is to replace the original deterministic equations by ones that include stochastic forcing. In case of Brownian motion, the original equations are replaced by Langevin-type equations.

Inspired by the statistical approach used for handling Brownian motion, Hasselmann proposed to describe climate variability using stochastic climate models (Hasselmann, 1976). These models are formulated for the slow components of x. In line with the statistical treatment of slow Brownian particles embedded in fast fluid molecules, a stochastic climate model for a slow component x is written as

$\overline{F}$ represents the slow dynamics of x and the averaged effect of the fast components of x on x, with $\overline{(\cdot )}$ being an average over a time period longer than the timescale of the fast components but shorter than the timescale of x. ζ is a stochastic forcing used to describe the fluctuating effect arising from the fast components.

Statistical approaches are efficient in constructing different variance-generation mechanisms. In case of Hasselmann’s stochastic climate model, a solution obtained by integrating Eq. (2) over time contains an integral of ζ over time, which is a random walk. The variance of a random walk increases with increasing time. In order to obtain a stationarily varying solution from Eq. (2), $\overline{F}$ must incorporate negative feedbacks (Hasselmann, 1976). Thus, variations generated by a stochastic climate model result from the joint effect of random-walk and negative feedbacks. Statistical approaches can also be accurate in describing random behaviors, if the stochastic forcing is carefully constructed to possess specific properties. What statistical approaches do not explicitly address is the mechanism responsible for the randomness in solutions of the considered system.

The other type of uncertainty arises from our inability to specify the exact initial conditions from which the considered physical system starts to evolve with time. This problem, first described by Lorenz (1963) and well-known to the numerical weather forecast community, is one of the key aspects studied by the dynamical systems theory. There, the sensitivity to initial conditions is attributed to the chaos arising from non-linear dynamics in a dynamical system. However, dynamical systems theory does not explicitly deal with randomness. It is unclear whether and to what extent chaotic solutions are random.

Quite the contrary, both the statistical approaches for handling high-dimensional systems and the investigation addressing the sensitivity to initial conditions implicitly assume that a physical system is fundamentally deterministic. The situation is understandable, since the uncertainties, which represent randomness, do not originate from the deterministic dynamics. Instead, they result solely from external factors related to our inability in tracking the exact solution or in specifying precise initial conditions. This assumption about determinism is in obvious conflict with the randomness which we experience from physical systems.

One step towards resolving this conflict is made by the finding that the determinism, as dictated by Eq. (1), breaks down under certain circumstances (von Storch, 2022). Given Eq. (1), the spectra of x and F, Γ^x(ω) and Γ^F(ω) where ω is frequency, are related to each other via

Eq. (3) seems to confirm the determinism that variations of x at any one frequency must be generated by the variations of F at the same frequency. This however cannot be true for a solution whose spectrum Γ^x(ω) is continuous and approaches a finite and non-zero Γ^x(0) as ω → 0. Given a finite and non-zero Γ^x(0), Eq. (3) requires that Γ^F(ω) must go to zero as ω → 0 so that Γ^F(0) = 0. Thus, at frequency ω = 0, variations of x can not be generated by variations of F at this frequency.

Before elaborating the meaning of the just mentioned low-frequency shape of Γ^x(ω), we point out that the determinism described by Eq. (3), which holds for all frequencies except zero frequency, is the norm that can become more prominent in case when F contains a time-varying external forcing. The present paper does not question and is not concerned with this determinism. To concentrate on internal dynamics, in which the origin of randomness presumably lies, we will focus on physical systems that are not influenced by any time-varying external forcing. We cannot rule out the presence of constant external forcings, as variations in a physical system, no matter random or deterministic, necessitate external power support.

Come back to the low-frequency shape of Γ^x(ω). That Γ^x(ω) is continuous and has finite and non-zero spectral value as ω → 0 describes nothing other than the manifestation of randomness in the solution of x. When defined as a Fourier cosine transform of auto-covariance function, the spectrum of a solution Γ^x(ω) only exists when the auto-covariance function is absolutely summable. Upon existence, Γ^x(ω) must be continuous, since a cosine function is continuous and since a Fourier cosine transform is a sum of weighted cosine functions. The condition of absolute summability implies that the auto-covariance function must decay to zero with increasing time lag. It is precisely this decay of auto-covariance function that diminishes serial correlation and makes a solution to appear random. It is also this decay of auto-covariance function that prohibits the solution of x to be purely periodic. Auto-covariance function of a purely periodic solution, whose spectrum consists of distinct spectral lines (Priestley, 1981), does not decay and retain its magnitude as time lag increases. It is still this decay of auto-covariance function, that allows Γ^x(0), the value of Γ^x(ω) at ω = 0, to be finite and non-zero. To see this, note that being a Fourier cosine transform of an auto-covariance function and since the value of a cosine function at the origin is one, Γ^x(0) is identical to the sum over the auto-covariance function at all time lags. Given that an auto-covariance function has a positive maximum at lag zero, the sum of an auto-covariance function that decays with increasing time lag can lead to a Γ^x(0) that is not zero and finite. The same argument does not apply to Γ^F(0), since auto-covariance function of F consists of differences of auto-covariance function of x because of Eq. (1) (von Storch, 2022).

The finite and non-zero low-frequency shape of Γ^x(ω) can be inferred from spectra of variables that are apparently random. Figure 1 shows a collection of such spectra. To this end we note that while the deterministic influence of external forcing can be easily controlled in a numerical experiment, achieving the same for the real climate is challenging. The real climate is subjected to an external forcing, that has a non-zero mean and varies with time. The real climate can hence reveal not only random behaviors resulting from internal dynamics (via e.g. instability and turbulence), but also deterministic behaviors resulting from external forcing. The latter includes for example long-term trends as responses to a slowly varying external forcing, and oscillations (e.g. annual cycle) as responses to a periodic external forcing. Thus, if we want to find from observations spectra that are continuous and have finite and non-zero values at the lowest frequencies, we need to consider those variables whose variations are mainly generated by internal dynamics, with the influence of external forcings being negligibly small relative to that of these internal dynamics.

Figure 1

Figure 1a) shows a spectrum of a component x of a dry atmospheric model (James & James, 1989), generated by model’s internal dynamics without influence of any time-varying external forcing. Figure 1b) shows spectra of sea level pressure derived from an atmospheric reanalysis (Deser et al., 2012) (black lines). Figure 1d) and e) show spectra of current kinetic energy derived from instrumental records (Ferrari & Wunsch, 2009). We assume that sea level pressure and ocean current are variables whose variations arise mainly from internal dynamics. All these spectra reveal finite and non-zero values at the lowest resolved frequencies. Finally, Figure 1c) shows the spectra derived from the Lorenz model (Lorenz, 1963), a model that does not contain any time-varying external forcing. In contrast to the other spectra depicted in Figure 1, which are merely indicative owing to the limited duration of available observations and model solutions, the finite and non-zero low-frequency shape of Γ^x(ω) can now be demonstrated asymptotically by considering longer and longer Lorenz solutions (von Storch, 2022). We conclude that for a solution of a system governed by a set of equations in form of Eq. (1), the apparent randomness is manifested in the solution’s spectrum that is continuous and has a finite and non-zero Γ^x(0). This spectral feature enforces the breakdown of determinism at zero frequency, including the associated asymptotic behavior towards the breakdown at near-zero frequencies. The breakdown suggests that Γ^x(0) has nothing to do with F, which is puzzling at first glance.

On further reflection, we notice that the randomness in x and the wholly deterministic nature of F do not pertain to the same thing. Randomness in x is only evident when a solution of x at time t is set in relation to the solution of x at a distant time t+T with T ≠ 0. A Brownian particle appears to move randomly because its velocity at time t seems to be independent of its velocity at time t+T, where T is a time interval larger than the reaction time of the human eye. F on the other hand tells us about the evolution tendency. Given F of the velocity of a Brownian particle at time t (which is a function of the whole state vector x describing the positions and velocities of all involved particles and molecules at time t), the time rate of change of the velocity of the considered particle is know exactly. Nothing is random. Thus, if we want to understand the mechanism behind the randomness, we should shift to examining the integral forcing G_T, a definite integral of F(x(t)) over a time span of length T that drives the evolution from x(t) to x(t+T). Studying G_T contrasts with the standard approaches that emphasize solely the differential forcing F.

This paper explores the properties of G_T(t). Following some preliminaries provided in Section 2, we show in Section 3 that G_T consists of a fluctuating and a dissipating components, in accordance with the fluctuation-dissipation theorem of Callen and Welton (1951). This theorem was introduced to the realm of climate research by Leith (1975) who showed how the theorem can be used to estimate climate responses to a changing external forcing, no matter which component x of x is considered. We show that there exists a threshold such that for T larger than this threshold, G_T emerges as a unified forcing. Section 4 describes the impacts of these properties of G_T on the solution of x. Section 5 and 6 discuss two aspects that are essential for the dissipation represented by G_T. Conclusions are provided in Section 7.

2 Preliminaries

3 Properties of integral forcing

Important for the consideration below is the condition of a physical system referred to as equilibrium. This condition can be achieved under the influence of constant external forcings. For a multi-dimensional system, an equilibrium is generally not described by a solution that is independent of time, but by a solution that varies stationarily with time. If the external influences were kept constant forever, the solution would continue to vary stationarily into infinite times. In case of a climate model, an equilibrium of the model can be reached by integrating the model under constant external forcing conditions for some time (to allow the model to spin up).

Consider a multi-dimensional system in equilibrium. For every component x of the system’s state vector x, and for any $\tau \in {\mathbb{Z}}_{+}$, the properties of the integral forcing of x, $G_{\tau ,i\tau }\in \left\{G_{\tau ,i\tau }\right|i\in {\mathbb{Z}}_{*}\}$, are described by the following three postulates.

• I. G_τ,iτ consists of, apart from a constant ${\widehat{c}}_{\tau }$, a dissipating component ${\widehat{d}}_{\tau }{x}_{i\tau }$ and a fluctuating component f_τ,iτ, and can be written as

${\widehat{c}}_{\tau }$ and ${\widehat{d}}_{\tau }$ are the intercept and the slope of the line obtained by regressing G_τ,iτ against x_iτ using n pairs of (x_iτ,G_τ,iτ) along a solution, where n is finite. f_τ,iτ, the residual not described by the regression line, is determined such that G_τ,iτ in Eq. (9) is identical to G_τ,iτ in Eq. (8) calculated from Eq. (7).

• II. The expression given in Eq. (9) is unique in the sense that it can be replaced by

where

Moreover, the dissipating and fluctuating components are related to each other via

where ${\sigma }_{f_{\tau }}^2$ is the variance of the series $\left\{f_{\tau ,i\tau }\right|i\in {\mathbb{Z}}_{*}\}$ and ${\sigma }_x^2$ is the variance of the series $\left\{x_{i\tau }\right|i\in {\mathbb{Z}}_{*}\}$. On the plane spanned by d_τ and ${\sigma }_{f_{\tau }}^2$ or the plane spanned by d_τ and ${\sigma }_{f_{\tau }}^2\text{/}{\sigma }_x^2$, Eq. (12) is a curve that has its maximum at the center where d_τ = –1 and is mirror symmetric about d_τ = –1. Such a curve is referred to as a fluctuating-dissipating curve, or for short a fd-curve.

• III. There exists a threshold τ₀ such that G_τ,iτ with τ > τ₀ emerges as a unified forcing consisting of a dissipating component characterized by d_τ = –1, and a fluctuating component f_τ,iτ that behaves like a white noise.

Postulate I, which is the basis of all postulates, adopts the idea behind the fluctuation – dissipation theorem (Callen & Welton, 1951) that for a system in equilibrium, anything that generates fluctuations must also damp the fluctuations. In case of the Brownian motion, the collisions with fluid molecules make a Brownian particle to fluctuate. At the same time, the collisions introduce a drag that damps the movement of the particle. Postulate I says that for a system in equilibrium, G_τ always contains a dissipation, independent of the value of τ and no matter which one of the components of x is considered. Whether this is true is a priori not clear.

To verify these postulates, we need many long series {x_iτ|i = 1,2,…} and {G_τ,iτ|i=1,2,…}, for many different values of of τ. Despite the advance of computer technology, numerically deriving all these long series is still challenging for a high-dimensional system, such as a climate model or a Brownian system. We hence verify these postulates in terms of the Lorenz model (Lorenz, 1963). This model is multi-dimensional and possesses an equilibrium described by stationarily varying and seemingly random solutions.

3.1 Verification of Postulate I

Formally, G_τ,iτ can always be described by the expression given in Eq. (9) using a properly chosen f_τ,iτ. Since no conditions have been imposed on f_τ,iτ, apart from its existence, Postulate I is verified by showing that ${\widehat{d}}_{\tau }$ is negative for all components of x and for all τ ≥ 1. Figure 2 shows for the three Lorenz components (magenta, blue and green) and for five values of τ that the regression line is indeed always tilted with a negative slope. The exact values of ${\widehat{d}}_{\tau }$ are truncated to two digits after the dot and listed in the bottom left corner of each scatter diagram. Negative slopes are also found for all other considered values of τ, as shown by Figure 5.

Figure 2

3.2 Verification of Postulate II

Postulate II is verified in terms of Figures 3 and 4. Figure 3 shows for all three Lorenz components and for two different values of τ that ${\widehat{c}}_{\tau }$ and ${\widehat{d}}_{\tau }$ converge with increasing n, the number of data points $(x_{i\tau },G_{\tau ,i\tau })$ used for calculating the regression line. The convergences suggest that both $c_{\tau }=\underset{n\to \infty }{\lim }{\widehat{c}}_n$ and $d_{\tau }=\underset{n\to \infty }{\lim }{\widehat{d}}_n$ exist. G_τ,iτ can hence be uniquely expressed in terms of Eq. (10). The notions ${\widehat{c}}_{\tau }$ and ${\widehat{d}}_{\tau }$ are still used, since everything we show are derived from a finite number of data points along a solution.

Figure 3

Figure 4

Figure 4 shows the fd-curves complemented by the variances of the three Lorenz components (black lines). For all three Lorenz components, the points $({\widehat{d}}_{\tau },{\widehat{\sigma }}_{f_{\tau }}^2)$ (magenta, blue, and green dots) are located right on the fd-curve ${\sigma }_{f_{\tau }}^2={\sigma }_x^2(1-{(1+d_{\tau })}^2)$ (top); and the points with normalized variance, $({\widehat{d}}_{\tau },{\widehat{\sigma }}_{f_{\tau }}^2\text{/}{\widehat{\sigma }}_x^2)$, are located right on the fd-curves ${\sigma }_{f_{\tau }}^2\text{/}{\sigma }_x^2=(1-{(1+d_{\tau })}^2)$ (bottom). Thus, the relation between ${\widehat{\sigma }}_{f_{\tau }}^2$ and ${\widehat{d}}_{\tau }$ can be readily described by Eq. (12) for a large but finite n. Appendix B shows further how Eq. (12) emerges in the limit n → ∞.

Regarding the points $({\widehat{d}}_{\tau },{\widehat{\sigma }}_{f_{\tau }}^2)$ or $({\widehat{d}}_{\tau },{\widehat{\sigma }}_{f_{\tau }}^2\text{/}{\widehat{\sigma }}_x^2)$, there is a difference between the three Lorenz components. As τ increases, the points of the first two Lorenz components (magenta and blue dots) move from the right end to the center of the fd-curve, and eventually stay and remain to stay at the center of the curve. ${\widehat{d}}_{\tau }$ strengthens monotonically from zero to –1 with increasing τ, and equals –1 for τ larger than a threshold. For the third Lorenz component, the points (green dots) move with increasing τ from the right end of the curve toward the left, pass the center of the curve, and reach the most left position at ${\widehat{d}}_{\tau }>-2$. As τ further increases, they move backward toward the right, pass the center of the curve, and reach the most right position at ${\widehat{d}}_{\tau }<0$. Thereafter, they continue to move back and forth around the center of the fd-curve, with the far left and the far right position reached becoming increasingly close to the center the fd-curve. As a result, ${\widehat{d}}_{\tau }$ strengthens from zero to –1 in a non-monotonic manner.

3.3 Verification of Postulate III

Postulate III is verified by Figures 5 and 6. The two figures show that for all three Lorenz components, there exists a threshold τ₀ such that for τ > τ₀, G_τ,iτ represents a unified forcing. The phrase “unified” refers to the same type of G_τ,iτ, no matter which component of x is considered, and independent of values of τ provided τ > τ₀. This unified forcing contains a dissipating component that is characterized by d_τ = –1 and a fluctuating component whose auto-correlation function resembles that of a white noise. The threshold τ₀, beyond which the unified forcing is found, depends on the component x considered. It is smaller for the first two Lorenz components (magenta and blue) than for the third Lorenz component (green).

Figure 5

Figure 6

By definition, G_τ,k is the sum over F_j at τ time steps obtained when integrating the whole system from time step k to time step k+τ –1. Before G_τ,k with d_τ = –1 is produced, the forward integration first produces G_1,k = F_kΔt, then G_2,k = F_kΔt+F_k+1Δt, and so forth, and eventually $G_{\tau ,k}={\Sigma }_{j=k}^{k+\tau -1}{F}_j\Delta t$. Thus, we should see a general strengthening of the dissipating component, characterized by an overall increase from |d₁|, to |d₂|, and so forth, before the maximum characterized by |d_τ| = 1 is reached. A sign of this can already be seen from Figure 2, which shows a general strengthening of d_τ with increasing value of τ (from top to bottom row in Figure 2).

For τ < τ₀, the way how the dissipating component of G_τ,iτ strengthen with increasing τ is different for different Lorenz component. While the strengthening is monotonic for the first two Lorenz components, it is non-monotonic for the third Lorenz component. The former is characterized by the uni-dimensional movement of the $({\widehat{d}}_{\tau },{\widehat{\sigma }}_{f_{\tau }}^2)$-point along the fd-curve with increasing τ described before, which results in the magenta and blue lines in Figure 5. The latter is characterized by the back and forth swing of the $({\widehat{d}}_{\tau },{\widehat{\sigma }}_{f_{\tau }}^2)$-point along the fd-curve, which results in the green lines in Figure 5.

4 Impacts of integral forcing G_τ

The dissipation, that is associated with G_τ and characterized by d_τ, is the same between any two adjacent members in the series $\left\{x_{i\tau }\right|i\in {\mathbb{Z}}_{*}\}$. As such, it systematically weakens the link between x_iτ and x_iτ+τ, resulting in an auto-correlation function of x at lag τ, ρ_τ, whose magnitude is smaller than one. This relation between d_τ and ρ_τ (see Appendix C for its derivation) is described by

Although presented as an equality, ρ_τ should be regarded as the effect resulting from d_τ, since x_iτ+τ that has a weaker link to x_iτ is generated by G_τ,iτ that diminishes x_iτ by the amount quantified by |d_τ|.

For G_τ with τ > τ₀, G_τ,iτ = c_τ+ d_τ x_iτ +f_τ,iτ is replaced by

with f_τ,iτ being a white-noise-like forcing. With Eq. (14), Eq. (8) reduces to

x_iτ+τ becomes independent of x_iτ, a behavior deemed as random in Section 1. We hence conclude that it is the integral forcing G_τ of x with τ > τ₀, that makes the solution of x to become random. Given Eq. (15), the variance of the series $\left\{x_{i\tau }\right|i\in {\mathbb{Z}}_{*}\}$, which equals also the variance of the series $\left\{x_j\right|j\in {\mathbb{Z}}_{*}\}$, becomes identical to the variance of $\left\{f_{\tau ,i\tau }\right|i\in {\mathbb{Z}}_{*}\}$. Consequently, the ratio $r={\sigma }_{f_{\tau }}^2\text{/}{\sigma }_x^2$ is identical to one, as shown in Figure 5b).

Furthermore, for any two adjacent members in the series $\left\{x_{i\tau }\right|i\in {\mathbb{Z}}_{*}\}$, it is impossible to determining the past member x_iτ from the future member x_iτ+τ, despite of Eq. (8). This is because as G_τ,iτ with τ > τ₀ emerges through forward integration, the dissipating component of G_τ,iτ cancels with the past state x_iτ little by little and eventually completely, at the time when x_iτ+τ is generated by the fluctuating component f_τ,iτ of G_τ,iτ at time step iτ+τ. Consequently, x_iτ is independent of f_τ,iτ. This independence leads to a parallelogram-like shape of the scatter obtained when regressing G_τ,iτ against x_iτ (two bottom rows of Figure 2). The evolution from x_iτ to x_iτ+τ is not only random but also irreversible.

The relation between two adjacent members in the series $\left\{x_{i\tau }\right|i\in {\mathbb{Z}}_{*}\}$ with τ > τ₀ is in striking contrast with the relation between x(t) and x(t+δt) with an infinitesimal δt for a continuous solution, or the relation between x_k and x_k+1 for a discrete solution. Given F(x(t)), Eq. (1) is also valid when time is reversed. Given F_k, Eq. (6) can be integrated for a given past state x_k forward in time to predict the future state x_k+1, or integrated for a given future state x_k+1 backward in time to predict the state member x_k. The evolution from x(t) to x(t+δt) with an infinitesimal δt is reversible, so does the evolution of from x_k to x_k+1. The key to the reversibility is the differential forcing F(x(t)), or F_k, which represents a forcing rate at a time instant. This stands in stark contrast to G_τ,k, which is a forcing over a time span of non-zero length.

5 Significance of passing of time

A further aspect that makes G_τ different from F concerns the dissipation represented by G_τ, which should not be confused with the damping included in F. We refer the latter as “damping” to distinguish it from the dissipation in G_τ. In the Lorenz model, F contains a linear damping ax with a = –10, –1, and –8/3 for the three components respectively. The damping in F differs from the dissipation in G_τ. Being a differential forcing, the strength of the damping (i.e. a in the Lorenz model) represents a damping rate, and has the unit of 1/[t], with [t] being the unit of time. Different from that, the dissipation in G_τ, which is characterized by d_τ, represents a portion of dissipation and is dimensionless. More importantly, the damping in F_j is not associated with any specific timescale, consistent with the fact that it represents a rate, whereas d_τ is associated with one and only one timescale of length τΔt. d_τ represents the dissipation experienced by an evolution of x from x_k to x_k+τ over τ time steps.

We further explore the difference between the damping in F and the dissipation in G_τ using the Lorenz model. In this paper, the Lorenz model is solved using Δt = 0.01. With this value of Δt, the damping within one time step, aΔt, equals –0.1, –0.01, and –0.027 for the three Lorenz components, respectively. Here, we have disregarded the impact of the numerical scheme used for solving the discretized equations, which can affect the damping amount by a few percent. The values of aΔt can be compared with the dissipation experienced by x as x evolves from x_i to x_i+1 over a time span of length Δt, which is quantified by d₁ and listed in the first row of Figure 2. We find that the values of aΔt are much larger than the values of d₁.

We further explore the difference between the damping in F and the dissipation in G_τ by considering the limit Δt → 0. In this limit, Eq. (6) converges to Eq. (1), and d₁ converges to d_T with T = 0 defined for a continuous solution. Since for T = 0, G_T(t) = G₀(t) = 0 for all $t\in {ℝ}_{*}$ d_T with T = 0 must also be zero. However, the fact that d₁ → 0 in the limit Δt → 0 does not make d₁ so different from the damping within one time step, aΔt, since we have also aΔt → 0 in the limit Δt → 0. The difference between the damping in F and the dissipation in G_τ becomes only apparent when considering the rate of dissipation and the rate of damping. Figure 7 shows that aΔt is proportional to –Δt, whereas d₁ is proportional to –Δt². Thus, the dissipation rate vanishes,

whereas the damping rate

is generally not zero. Eq. (16) and Eq. (17) suggest that the damping in F and the dissipation in G_τ are two different things. The dissipation in G_τ cannot be included in F as a forcing rate, since this rate vanishes exactly.

Figure 7

Given the link of d_τ to the specific timescale of length τΔt, we may interpret the dissipation in G_τ,k as something that results from interactions of x_k with other components of x, that have taken place within a time span covering τ time steps starting from the k-th time step. The length of the time span is τΔt. For τ = 1 and when Δt goes to zero, the length of the time span, τΔt = Δt, goes to zero. No interaction of x with other components of x can complete within a time span of vanishing length. d₁ approaches zero. On the other hand, increasing the value of τ for a given Δt increases the length of time span τΔt. The larger the value of τ, the more interactions between x_k and other components of x can take place within the time span extending from the k-th to the (k+τ-1)-th time step, the stronger is the dissipation resulting from these interactions. The threshold τ₀, beyond which G_τ equals the unified forcing, corresponds to the length of the time span that starts from the k-th time step and encompasses all interactions, and only these interactions, between x_k and other components of x. Further extending the length of this time span (by increasing τ) allows more interactions to occur within the time span. However the additional interactions no longer involve x_k at the k-th time step and hence no longer contribute to the dissipation of x_k.

Accepting the idea of the fluctuation – dissipation theorem that for a system in equilibrium, anything that generates fluctuations must also damp the fluctuations, this “anything” is manifested in actions that take place in form of interactions of x with other components of x. Without the passing of time, these actions cannot be completed and the associated dissipation cannot take effect. The demand on the passing of time is in stark contrast to the damping in F, which is a forcing rate needed to balance the rate of external forcing, and exists without the passing of time.

6 Significance of multi-dimensionality

The interpretation of the timescale dependence of d_τ suggests that multi-dimensionality is a necessarily condition for G_τ to possess a dissipation that allows Postulate III to be valid, and with that a solution that is random. Even though we are unable to prove this assertion rigorously, we provide below some supporting evidences. We do so by considering two one-dimensional systems as counterexamples, for which Postulate III is not valid, and consequently whose solutions cannot be random.

The first example is the one-dimensional system $\frac{\text{d}x}{\text{d}t}=\beta$, where β is a constant. This system has the analytical solution x(t) = x₀+βt. The differential forcing of x is β; the integral forcing of x is G_T(t) = βT. For a given non-zero value of T, the regression slope d_T obtained from regressing G_T(t) against x(t) is zero, since G_T(t) is independent of t and then independent of x(t), no matter whether β is positive or negative. With d_T = 0, G_T(t) does not contain a dissipating component. Postulate I is not valid. Without Postulate I, the other two postulates, especially Postulate III, are meaningless. The solution x(t) = x₀+βt is always deterministic.

The second example is the one-dimensional cosine model, $\frac{\text{d}x}{\text{d}t}=\cos (2\pi t\text{/}P)$ with period P. This model has the analytical solution $x(t)=x_0+\frac{P}{2\pi }\sin (2\pi t\text{/}P)$. The differential forcing of x is cos(2πt/P); the integral forcing of x is $G_T(t)=\frac{P}{2\pi }(\sin (2\pi (t+T)\text{/}P)-\sin (2\pi t\text{/}P))$. Figure 8 shows for six values of T and for t = iT and i = 1,…,n, how G_T(t) are scattered against x(t). Also shown are the regression line G_T(iT) = c_T+d_T x_iT + f_T,iT for each value of T. In all six cases, the regression lines are tilted with a slope d_T < 0, albeit d_T with a value of T that is close to a multiple of P (as in Figure 8a, f) is close to zero and has to be listed as –0.00 when keeping only two digits after the point. The negative slope is also found for T = P/4 (Figure 8d), for which the period of G_T(iT) is four and the regression line goes through only four pairs of (x_iT,G_T(iT)). Thus, the integral forcing G_T(iT) can also be decomposed into a dissipating and a fluctuating component for a periodic solution. Postulate I is valid for the cosine model. The idea that for a system in equilibrium, anything that generates fluctuations must also dampen those fluctuations, seems to apply universally to all types of stationarily varying solutions, regardless of whether they are periodic or non-periodic.

Figure 8

Postulate II is also valid for the cosine model. The points $(d_T,{\sigma }_{f_T}^2)$ (black dots in Figure 4), which can be calculated using the analytical expressions of G_T(iT) and x(iT) with i=1,…, n, are located right on the corresponding fd-curve, which is indicated by the orange line in Figure 4a) and collapses to the black line in Figure 4b). Thus, the dissipating and fluctuating component of the integral forcing of a periodic solution are also related to each other via Eq. (12).

The situation is different for Postulate III. The general strengthening of d_T with increasing T, which in this example can only occur in a non-monotonic manner, cannot be realized by the one-dimensional cosine model. d_T, which is a periodic function of T, retains its overall strength with increasing T. The points $(d_T,{\sigma }_{f_F}^2)$ or $(d_T,r_T)$ (black dots in Figure 4) swing with increasing T from the right end (where d_T = 0) to the left end (where d_T = –2) of the fd-curve and continue to swing with the same reach as T goes to infinity. No threshold of T exists such that for T larger than this threshold, G_T(t) reduces to a forcing consisting of a dissipating component with d_T = –1 and a white-noise like fluctuating component. Postulate III is not valid. The sinus solution at time t is always related to the sinus solution at time t+T later, independent of the value of T.

7 Conclusions

Consider a system described by a multi-dimensional state vector x, whose evolution is governed by a set of equations in form of dx/dt = F(x(t)) with x being a component of x and F = F (x(t)) being a deterministic function of x. When solving such a system at discrete time steps, the solution of x at a time step can become independent of the solution of x at a later time step, a behavior deemed as random. This paper examines how this randomness arises from internal dynamics represented by F. We do so by exploring the properties of the integral forcing G_τ,k, which equals the integral over F at τ time steps starting from the k-th time step. G_τ,k is responsible for the evolution of x from x_k to x_k+τ. The following conclusions are drawn.

First, for a system in equilibrium, the integral forcing G_τ,k consists of (apart from a constant c_τ) a dissipating component d_τx_k with d_τ < 0 and a fluctuating component f_τ,k, and can be expressed as G_τ,k = c_τ+d_τ x_k+f_τ,k. This expression is in accordance with the idea behind the fluctuation – dissipation theorem that for a system in equilibrium, anything that generates fluctuations must also damp the fluctuations. The two components of G_τ,k are related to each other following the rule described by the fd-curve. There exists a threshold τ₀ such that G_τ,k with τ > τ₀ emerges as a unified forcing. The dissipating component of this forcing is characterized by d_τ = –1, and the fluctuating component of this forcing behaves like a white noise, independent of τ, as long as τ > τ₀, and no matter which component of x is considered.

Second, for τ > τ₀, the state x_k+τ, which is nominally produced by G_τ,k via x_k+τ = x_k+G_τ,k, equals then x_k+τ = c_τ+f_τ,k, with f_τ,k being a white-noise-like forcing. The series $\left\{x_{i\tau }\right|i\in {\mathbb{Z}}_{*}\}$ becomes random, since any one member in the series is independent of any other member of the series. This series is also irreversible, since a member x_iτ is little by little canceled by the dissipation that emerges as soon as the system is integrated forward in time. By the time when the system is integrated over τ time steps to allow the emergence of G_τ,iτ, x_iτ is completely canceled by the dissipating component of G_τ,iτ. x_iτ+τ is generated by the fluctuating forcing of G_τ,iτ, which is independent of x_iτ.

Third, while the damping in F_j represents a typically non-zero damping rate needed for counterbalancing the rate of external forcing, the dissipation in G_τ,k arises from actions completed over a time span of non-zero length. More precisely, these actions are interactions of x_k with other components of x completed during the time span extending from time step k to time step k+τ–1. The number of these interactions inevitably goes to zero when the length of the time span goes to zero. It reaches a maximum, when the length of the time span equals τ₀ time steps. Since the completion of these actions requires the passing of time, the resulting dissipation cannot be included in the differential forcing F.

Finally, being arising from interactions among components of x, randomness is a peculiar feature of a multi-dimensional system. The solution of a one-dimensional system cannot be random.

The above conclusions are drawn based on the integral forcing numerically obtained from the Lorenz’s 1963 model. Verifying them for high-dimensional systems requires great computational efforts. By suggesting that G_τ consists of a dissipating and a fluctuating component, we link the mechanism responsible for the emergence of randomness with the fluctuation-dissipation theorem known in statistical physics. By demonstrating that the dissipation in G_τ cannot be included in F but emerges as soon as the system is integrated forward in time, we identify the mechanism as resulting from interactions completed within a time span of non-zero length. When further verified, the idea behind the fluctuation and dissipation theorem should be considered as generally valid for multi-dimensional systems that are in equilibrium and governed by differential equations in form of dx/dt = F.

Appendix A: Calculation of intercept c_τ, regression slope d_τ and residual f_τ,iτ

This appendix shows how the intercept c_τ, the regression slope d_τ, and the residual f_τ,iτ (or the fluctuating component of G_τ,iτ) and its variance ${\sigma }_{f_{\tau }}^2$ are calculated. Since Eq. (9) represents a regression of G_τ,iτ against x_τ,i, we use the known result of least squared fitting and define

Given ${\widehat{c}}_{\tau }$ and ${\widehat{d}}_{\tau }$, f_τ,iτ is defines as

with variance

We use ^ to distinguish quantities obtained from a finite number n of data points from quantities obtained in the limit n → ∞:

and

For the Lorenz model, ${\widehat{c}}_{\tau }$ and ${\widehat{d}}_{\tau }$ (Figure 3) and ${\widehat{\sigma }}_{f_{\tau }}^2$ (not shown) converge with increasing value of n.

Appendix B: Derivation of the fd-curve

This appendix derives the fd-curve that describes the relation between the dissipating and fluctuating component of an integral forcing G_τ,iτ with $\tau \in {\mathbb{Z}}_{+}$. We start from expressing G_τ,iτ in terms of intercept ${\widehat{c}}_{\tau }$, regression slope ${\widehat{d}}_{\tau }$ and residual ${\widehat{f}}_{\tau ,i}$, defined using n data points along a solution, with n being finite, and proceed further by considering the limit n → ∞.

For $\tau \in {\mathbb{Z}}_{+}$, we rewrite Eq. (8) using Eq. (A3) as

We define the mean and the variance of the series {x_iτ|i=1,…, n} by

and the mean of $\left\{{\widehat{f}}_{\tau ,i\tau }\right|i=1,\cdots ,n\}$ by

and the covariance between ${\widehat{f}}_{\tau ,i\tau }$ and x_iτ by

where

Rearranging Eq. (B1) by expressing x_(i+1)τ in terms of $x_{(i+1)\tau }-{\widehat{\mu }}_{x_{\tau }}$ and x_iτ in terms of $x_{i\tau }-{\widehat{\mu }}_{x_{\tau }}$ through adding and subtracting ${\widehat{\mu }}_{x_{\tau }}$, we find,

Squaring Eq. (B7) and applying ${\overline{(\cdot )}}^n$ to the result, we obtain after making use of ${\overline{x_{i\tau }-{\widehat{\mu }}_{x_{\tau }}}}^n=0$,

with

For a sufficiently large n, ${\overline{{(x_{(i+1)\tau }-{\widehat{\mu }}_{x_{\tau }})}^2}}^n$ is well approximated by ${\overline{{(x_{i\tau }-{\widehat{\mu }}_{x_{\tau }})}^2}}^n={\widehat{\sigma }}_{x_{\tau }}^2$. Eq. (B8) reduces to

Figure B1 shows for the three Lorenz components and for τ = 2 and τ = 5000 respectively, how the three A-terms defined in Eq. (B9)–Eq. (B11) evolve with increasing number n of data points used for their calculations. A₁ and A₂ (first two rows) converge fast to zero with increasing n. A₃ (bottom panel) is numerically not distinguishable from zero for all considered values of n. Similar behaviors are found for other values of τ, including τ = 1. The three A-terms in Eq. (B12) can hence be considered to be zero for $\tau \in {\mathbb{Z}}_{+}$ for sufficiently large value of n. In the limit n → ∞, Eq. (B12) can, after making use of Eq. (A7) and

be rewritten as

Figure B1

That the limit Eq. (B13) is independent of τ can be easily demonstrated numerically.

Appendix C: Derivation of the relation between ρ_τ and d_τ

This appendix establishes the relation between auto-correlation function ρ_τ of x and d_τ associated with the integral forcing G_τ of x. For $\tau \in {\mathbb{Z}}_{+},\hspace{0.17em}{\rho }_{\tau }$ and the respective covariance function ${\gamma }_{\tau }$ are defined by

where ${\widehat{\mu }}_{x_{\tau }}$ is defined in Eq. (B2). Multiplying Eq. (B7) by $(x_{i\tau }-{\widehat{\mu }}_{x_{\tau }})$ and applying ${\overline{(\cdot )}}^n$ (for its definition, see Eq. (B6)) to the result yields

where the equality ${\overline{x_{i\tau }-{\widehat{\mu }}_{x_{\tau }}}}^n=0$ is used. Since ${\overline{{\widehat{f}}_{\tau ,i\tau }x}}^n$ can be shown to be numerically not distinguishable from zero, similar to A₃ discussed in Appendix B, we can set ${\overline{{\widehat{f}}_{\tau ,i\tau }x}}^n$ to zero. In the limit n → ∞, Eq. (C2) reduces then, after making use of Eq. (B13), to

Data Accessibility Statement

All data and subsequently the figures are generated by Matlab-scripts archived in the publication repository: https://hdl.handle.net/21.11116/0000-000D-C83B-0.

Competing Interests

The author has no competing interests to declare.