Chapter 1: Stochastic Processes

Cumulative Distribution Function (CDF)

The cumulative distribution function of a random variable \(X\) is defined as:

The cumulative distribution function of a random variable is always defined by measurability.

Properties:

• \(F_X\) is non-decreasing.
• \(\lim_{x \to -\infty} F_X(x) = 0\) and \(\lim_{x \to \infty} F_X(x) = 1\).

Probability Density Function (PDF)

The probability density function> is the derivative of the cumulative distribution function (when it exists):

Properties:

• \(p_X(x) \geq 0\) for all \(x\).
• \(\int_{-\infty}^{\infty} p_X(x) \, dx = 1\).

Expectation

The first important notion in the theory of probability is the expectation of a random variable noted with brackets \( \langle \, \rangle\). A random variable \(X\) being measurable with respect to the measure \(\mathbb{P}\), we may introduce the formal integral:

However, this form is not convenient for practical calculations. The following theorem gives the expectation in terms of an integral of a real-valued function.

Transfer Theorem

For a measurable function \(f: \mathbb{R} \to \mathbb{R}\), the expectation of the random variable \(f(X)\) is:

Variance and Other Moments

The moments of \(X\) are:

The first moment (\(n=1\)) called mean value is noted \(\mu_X= \langle X \rangle\).

The variance of \(X\) is:

The standard deviation is \(\sigma_X = \sqrt{\text{Var}(X)}\).

The root-mean-square is defined as \(X_\mathrm{RMS} = \sqrt{\langle X^2 \rangle}\). We have \(X_\mathrm{RMS}^2 = \sigma_X^2 + \mu_X^2\).

Joint Distribution Function

The joint cumulative distribution function of \(X\) and \(Y\) is:

Joint Probability Density

The joint probability density> function is the partial derivative of the joint cumulative distribution function:

Properties:

• \(p_{X,Y}(x,y) \geq 0\) for all \(x, \, y\).
• \(\int_{-\infty}^{\infty} \int_{-\infty}^{\infty} p_{X,Y}(x,y) \, dx dy = 1\).

Marginal Probability Density

The marginal probability density function of \(X\) is obtained by integrating the joint probability density function:

Statistical Independence

Two random variables \(X\) and \(Y\) are said independent if the events \(X \leq x\) and \(Y \leq y\) are independent. This reads:

or equivalently

Transfer Theorem

For a two-variables measurable function \(f: \mathbb{R}^2 \to \mathbb{R}\), the expectation of the random variable \(f(X,Y)\) is:

Moments, Correlation, and Covariance

The moments are:

The correlation between \(X\) and \(Y\) is:

The variables \(X\) and \(Y\) are said uncorrelated if \(R_{X,Y} = \langle X \rangle \langle Y \rangle\). Note that independent variables are uncorrelated but the converse is not true.

The covariance between \(X\) and \(Y\) is:

The correlation coefficient is:

If \(\rho_{X,Y} = 0\), \(X\) and \(Y\) are uncorrelated.

Definition

A random process is a two-variable function \(X: I \times \Omega \to \mathbb{R}\), where \(I \subset \mathbb{R}\) is an interval (usually time).

For each fixed \(t \in I\), \(X(t,\cdot) : \Omega \to \mathbb{R}\) is a random variable.

For each fixed \(\omega \in \Omega\), \(X(\cdot, \omega): I \to \mathbb{R}\) is a sample function (realization of the random process).

We simply write \(X(t)\) ignoring the dependence with \(\omega\).

Let fix \(t_1, \, t_2, \, \dots , t_n \in I\). Then \(X_i=X(t_i), \, i=1, \dots , n\) are \(n\) random variables. They constitute a random vector to which all definitions of the previous section apply. In particular, we define the joint probability distribution function \[ F_{X_1, \dots, X_n}(x_1, \dots, x_n) = \mathbb{P}(X_1 \leq x_1 \& \dots \& X_n \leq x_n) \] and the joint probability density function \[ p_{X_1, \dots, X_n}(x_1, \dots, x_n) = \frac{\partial^n F_{X_1, \dots, X_n}}{\partial x_1 \dots \partial x_n}(x_1, \dots, x_n) \]

Second-Order Process

A random process is of second order if \(\langle X(t)^2 \rangle < \infty\) for all \(t\).

The mean function is:

The correlation function is:

When we have two second order processes, we introduce the cross correlation function:

The covariance is:

Mean and correlation, cross-correlation, and covariance functions always exist for second order random processes.

Stationarity

A second-order process is stationary (or weakly stationary of order 2) if:

• \(\mu_X(t + a) = \mu_X(t)\) for all \(a\)
• \(R_{XX}(t_1+a, t_2+a) = R_{XX}(t_1, t_2)\) for all \(a\).

Two second order processes are said mutually stationary if they are stationary and if:

• \(R_{XY}(t_1+a, t_2+a) = R_{XY}(t_1, t_2)\) for all \(a\).

The mean function does not depend on time and the correlation depends only on the time delay \(\tau = t_2 - t_1\) and we write: \[ \mu_X = \mu_X(t), \quad R_{XX}(\tau) = R_{XX}(t_1, t_2), \quad R_{XY}(\tau) = R_{XY}(t_1, t_2) \]

Correlation Function Properties

For a stationary process:

• \(R_{XX}\) is real-valued.
• \(R_{XX}(\tau) = R_{XX}(-\tau)\) (even function).
• \(R_{XX}(0) \geq |R_{XX}(\tau)|\) for all \(\tau\).

For two mutually stationary processes:

• \(R_{XY}\) is real-valued.
• \(R_{XY}(\tau) = R_{YX}(-\tau)\).
• \(R_{XX}(0)R_{YY}(0) \geq |R_{XY}(\tau)|^2\) for all \(\tau\).

Ergodicity

A stationary process is ergodic if its statistical properties (e.g., mean, correlation) can be deduced from a single realization. More exactly:

for almost all realizations of the random process (the equalities are realized with probability one).

Power Spectral Density (PSD)

The power spectral density \(S_{XX}(\omega)\) of a stationary process \(X(t)\) is defined as:

where \(\mathcal{F} \left[ X_T \right](\omega)\) is the Fourier transform of the truncated process

Properties:

• \(S_{XX}(\omega)\) is real-valued.
• \(S_{XX}(\omega) = S_{XX}(-\omega)\) (even function).
• \(S_{XX}(\omega) \geq 0\) (non-negative function).

Practical calculation for ergodic processes:

In practice, we use a single realization \(x(t)\) of the process \(X(t)\) supposed to be ergodic. We split \(x(t)\) into \(N\) time windows \(x_i(t)\) of length \(T\) with a fixed percentage of overlapping (usually 50 %). The power spectral density of \(X(t)\) is then estimated with:

where \(w(t)\) is a window function (e.g., Hamming, Hanning, ...).

Wiener-Khinchin Theorem

The power spectral density and the correlation function form a Fourier pair:

In particular, substituting \(\tau=0\) into the second equation gives

White Noise

A white noise process has a constant power spectral density:

Its correlation function is a Dirac delta:

Cross-Spectral Density

For two mutually stationary processes \(X(t)\) and \(Y(t)\), the cross-power spectral density is:

where \(\mathcal{F} \left[ X_T \right](\omega)\) is the Fourier transform of \(X_T\) and \(\overline{\mathcal{F}}[Y_T](\omega)\) the conjugate of the Fourier transform of \(Y_T\).

Properties:

• \(S_{YX}(\omega) = \overline{S_{XY}}(\omega)\) (complex conjugate).
• \(S_{YX}(\omega) = \overline{S_{XY}}(-\omega)\).
• \(|S_{XY}(\omega)|^2 \leq S_{XX}(\omega) S_{YY}(\omega)\) (Cauchy-Schwarz inequality).

Wiener-Khinchin Theorem

The cross-power spectral density and the cross-correlation function form a Fourier pair:

In particular, substituting \(\tau=0\) into the second equation gives

Coherence

The coherence between \(X(t)\) and \(Y(t)\) is:

It satisfies \(0 \leq \gamma_{XY}^2(\omega) \leq 1\). If \(X\) and \(Y\) are linearly related, \(\gamma_{XY}^2(\omega) = 1\).

Mean Square Limit

A sequence of random processes \(X_n(t)\) converges to \(X(t)\) in the mean square sense if:

We write

Properties:

It is always possible to permute expectation and limit symbols.

In the left-hand side appears a classical limit in real numbers but in the right-hand side the limit must be understood in the mean-square sense.

Continuity of a Random Process

A random process is said continuous in \(t\) if

in the mean-square sense. It is continuous if it is continuous in all \(t\).

Properties:

• A stationary second order random process \(X(t)\) is continuous everywhere if and only if the real-valued function \(R_{XX}(\tau)\) is continuous in \(0\).
• If \(X(t)\) is continuous, then \(\mu_X(t)\) is continuous.

Derivative of a Random Process

The derivative \(\dot{X}(t)\) exists if the following limit exists in the mean square sense:

Properties:

• If \(X(t)\) is differentiable, it is continuous.
• If \(X(t)\) is derivable, then \(\mu_X(t)\) is derivable and \[ \langle \dot{X}(t) \rangle = \frac{d}{dt} \langle X(t) \rangle. \] It is possible to permute expectation and derivation symbols.
• Derivation is a linear operator \[ \frac{d}{dt} \left[ \alpha X(t) + \beta Y(t) \right] = \alpha \dot X(t) + \beta \dot Y(t) \]
• Rule for deriving a product. \[ \frac{d}{dt} \left[ f(t) X(t) \right] = f(t) \dot X(t) + \dot f(t) X(t) \] where \(f(t)\) is a deterministic and derivable function. Caution, this rule can't be generalized to a producxt of random processes because the product of second order processes is not necessarily a second order process.
• Cross-correlation involving derivatives: \[ R_{\dot{X} Y}(t_1, t_2) = \frac{\partial}{\partial t_1} R_{XY}(t_1, t_2) \] \[ R_{X \dot{Y}}(t_1, t_2) = \frac{\partial}{\partial t_2} R_{XY}(t_1, t_2) \] \[ R_{\dot{X} \dot{Y}}(t_1, t_2) = \frac{\partial^2}{\partial t_1 \partial t_2} R_{XY}(t_1, t_2) \]
• For mutually stationary random processes \[ R_{\dot{X} Y}(\tau) = - R_{X \dot{Y}}(\tau) \] \[ \langle \dot{X} Y \rangle = - \langle X \dot{Y} \rangle \] \[ \langle \dot{X} X \rangle = 0 \]
• Cross-spectrum and derivative. \[ S_{\dot{X} Y}(\omega) = - i\omega S_{X Y}(\omega) \] \[ S_{X \dot{Y}}(\omega) = i\omega S_{X Y}(\omega) \] \[ S_{\dot{X} \dot{Y}}(\omega) = \omega^2 S_{X Y}(\omega) \] and when \(X=Y\) \[ S_{\dot{X} \dot{X}}(\omega) = \omega^2 S_{X X}(\omega) \] \[ S_{\ddot{X} \ddot{X}}(\omega) = \omega^4 S_{X X}(\omega) \]
• Root-mean square of derivative. \[ \dot{X}_\mathrm{RMS}^2 = \langle \dot{X}^2 \rangle = \frac{1}{2\pi} \int_{-\infty}^\infty \omega^2 S_{X X}(\omega) \, d\omega \] \[ \ddot{X}_\mathrm{RMS}^2 = \langle \ddot{X}^2 \rangle = \frac{1}{2\pi} \int_{-\infty}^\infty \omega^4 S_{X X}(\omega) \, d\omega \]

Integral of a Random Process

The integral of \(X(t)\) over \([a, b]\) is defined as the mean square limit of Riemann sums:

for arbitrary partition \(a_0=a < a_1 < \dots < a_n=b\) of step \(\epsilon= \mathrm{max}_i (a_i - a_{i-1})\) and arbitrary points \(s_i \in \left[a_{i-1}, \, a_{i}\right[\).

Properties:

• If \(X(t)\) is continuous on \([a, \, b]\), it is integrable over \([a, \, b]\).
• If \(X(t)\) is integrable, then \(\mu_X(t)\) is integrable and \[ \left\langle \int_a^b X(t) \, dt \right\rangle = \int_a^b \langle X(t) \rangle \, dt. \] It is possible to permute expectation and integration symbols, even for non-stationary processes.
• Integration is a linear operator. \[ \int_a^b \alpha X(t) + \beta Y(t) \, dt = \alpha \int_a^b X(t) \, dt + \beta \int_a^b Y(t) \, dt \]
• For any \(c\) \[ \int_a^b X(t) \, dt = \int_a^c X(t) \, dt + \int_c^b X(t) \, dt \]
• If \(X(t)\) is conitnuous, then \(\int_a^x X(t) \, dt\) is derivable and \[ \frac{d}{dx} \int_a^x X(t) \, dt = X(t) \]
• If \(X(t)\) is integrable and has a primitive \(Y(t)\), then \[ \int_a^b X(t) \, dt = Y(b) - Y(a) \]