betaFeedback

WorksPhysics and Astronomy

Statistical Treatment, Fourier and Modal Decomposition

Abstract

These are lecture notes for the lecture "Statistical Treatment, Fourier and Modal Decompositions", given at the VKI Lecture series "Fundamentals and Recent Advances in Particle Image Velocimetry and Lagrangian Particle Tracking". The course was held at the von Karman Institute for fluid dynamics from 15 November to 18 November 2021. This lecture provides a guided tour through the processing of data acquired via image velocimetry. Far from being an exhaustive account of the field, which would require an entire course on its own, the scope is to provide a hands-on tutorial. This begins with basic statistical treatment, briefly reviews frequency and modal analysis, and conclude with more advanced research topics such as multiscale modal decompositions and nonlinear dimensionality reduction. The material covered should hopefully propel newcomers into the subject while remaining of interest to experienced practitioners. All the codes related to this lecture are made available on a github repository.

How to cite this work

1  @InProceedings{Mendez2020,
2    author = {Miguel A. Mendez},
3    title = {Statistical Treatment, Fourier and Modal Decompositions},
4    booktitle = {Fundamentals and Recent Advances in Particle
5    Image Velocimetry and Lagrangian Particle Tracking, VKI Lecture Series},
6    editor = {Discetti, S. and Mendez, M. A},
7    year = {2021},
8    publisher = {von Karman Institute},
9    ISBN = {978-2-87516-181-9},
10 }

Lecture Format and Organization

Modern image velocimetry produces data with spatio-temporal resolutions that were inconceivable a couple of decades ago [52, 1]. This lecture is about processing that data and reviews a broad range of techniques.

With the intent of being accessible while providing some hands-on experience, the lecture is cast a sequence of exercises in Python. Thus, these notes should be read while keeping a Python terminal at hand, ready to run the provided scripts. Python is nowadays the lingua franca of data processing and “soft computing”; it is an extremely powerful language, it is free, and it is easy to use. While the reader might not be an experienced programmer, it is hoped that the presented scripts are simple enough to be intelligible. Running and interacting with these is essential to the correct delivery of the presented material. Readers interest in learning Python for scientific computing are referred to Svein Linge [66]; Langtangen [27]; Johansson [24]; Paul J. Deitel [47].

All the presented tools will be applied to a specific time-resolved PIV dataset. The details on this dataset and the instructions to download it are provided in § 3. The provided exercises guide the reader to the implementation of the most important data processing operations in statistical analysis, frequency analysis and modal decompositions. The theoretical background is kept to a minimum, and the reader is urged to consult the provided references to learn more.

The first part, in § 3, covers some essential statistical treatment, primarily focusing on time series analysis. There we treat the dataset as a set of independent time series corresponding to the velocity signal in various positions. These time series are, of course, not independent. However, the methods presented in this section do not explore nor exploit their spatial dependency. The tools introduced in this section are also the oldest: these were the only ones covered in data processing courses before image velocimetry made space and time resolution easily accessible. The second part, in § 4, moves from a “local processing” to a “global processing”, i.e. including both space and time analysis. This takes us to modal analysis, and the section reviews the notion of flow modes and modal decompositions. These are introduced first for the familiar Discrete Fourier Transform and then for the two most popular data-driven decompositions: the Proper Orthogonal Decomposition (POD) and the Dynamic Mode Decomposition (DMD). We shall explore “pro” and “cons” of each with the hope that the reader quickly realizes that the distinction between “pro” and “cons” is meaningless: these are simply different tools that serve different purposes and answer different questions.

The last part, in § 6, covers two advanced research topics. The first topic is an extension of the previous methods while keeping their linear framework: a recent decomposition that breaks the dichotomy POD/DMD and proposes a bridge between the two. This is the multi-scale Proper Orthogonal Decomposition (mPOD), extensively described in Mendez et al. [37]; Mendez [34]. The second topic also extends the previous methods but brings the treatment to a nonlinear framework, with kernels functions defining nonlinear similarity metrics. This section touches briefly on nonlinear dimensionality reduction and manifold learning. These are well-known tools in machine learning [9] but are now gaining popularity also in fluid mechanics, offering new ways of looking at the dynamics of fluid flows. § 7 closes with some final remarks.

Collecting the Dataset

The selected test case is the flow past a cylinder of d=5 mmd = 5\ \mathrm{mm} diameter and L=20cmL = 20\mathrm{cm} length in transient conditions, with varying free stream velocity. The experiments were carried out in the L10 low-speed wind tunnel of the von Karman Institute, instrumented with a TR-PIV system from Dantec Dynamics. The details of the experiment are described in Mendez et al. [38]. The dataset can be downloaded by the provided script Get_Cylinder_DATA.py, and particularly the following lines:

import urllib.request
print(’Downloading Data PIV for Chapter 10... ’)
url = ’https://osf.io/47ftd/download’
urllib.request.urlretrieve(url, ’Ex_5_TR_PIV_Cylinder.zip’)
print(’Download Completed! I prepare data Folder’)
# Unzip the file
from zipfile import ZipFile
String=’Ex_5_TR_PIV_Cylinder.zip’
zf = ZipFile(String,’r’);
zf.extractall(’./DATA_CYLINDER’); zf.close()

The dataset consists of nt=13200n_t = 13200 velocity fields, sampled at fs=3kHzf_s = 3\mathrm{kHz} over a grid of 71×3071 \times30 points. The spatial resolution is approximately Δx=0.85mm\Delta x = 0.85\mathrm{mm}. A plot of an exemplary time step, e.g. time step 10, can be printed as follows:

import os
import matplotlib.pyplot as plt
from Functions_Chapter_10 import Plot_Field_TEXT_Cylinder
FOLDER=’DATA_CYLINDER’
Name=FOLDER+os.sep+’Res%05d’%10+’.dat’ # Check it out: print(Name)
Name_Mesh=FOLDER+os.sep+’MESH.dat’
n_s,Xg,Yg,Vxg,Vyg,X_S,Y_S=Plot_Field_TEXT_Cylinder(Name,Name_Mesh)
Name=’Snapshot_Cylinder.png’
plt.savefig(Name, dpi=200)

The figure produced from this script is shown in Figure 1. Three probes, denoted as P1, P2 and P3, are placed to extract time series data for the following exercises. The location of these probes is also shown in the plot.

Example snapshot for the dataset analyzed in this lecture

Figure 1. Example snapshot for the dataset analyzed in this lecture. This is the flow past a cylinder in transient condition from Mendez et al. [38]. In the following section, the time series obtained at three virtual probes P1, P2, P3 will be analyzed. The location of these probes is shown in the figure.

The following code extracts the time series of the velocity in the three locations:

#%% Get Time Series
import numpy
import numpy as np
# Define the location of the probes in the grid
i1,j1=2,4; xp1=Xg[i1,j1]; yp1=Yg[i1,j1]
i2,j2=14,18; xp2=Xg[i2,j2]; yp2=Yg[i2,j2]
i3,j3=23,11; xp3=Xg[i3,j3]; yp3=Yg[i3,j3]
# Prepare the time axis
n_t=13200; Fs=3000; dt=1/Fs
t=np.linspace(0,dt*(n_t-1),n_t) # prepare the time axis#
# Initialize Probes P1,P2,P3
P1=np.zeros(n_t); P2=np.zeros(n_t); P3=np.zeros(n_t);
# number of velocity points
#from Plot_Field_TEXT_Cylinder)
nxy=int(n_s/2)
for k in range(0,n_t):
    # Name of the file to read
    Name=FOLDER+os.sep+'Res%05d'%(k+1)+'.dat'
    # Read data from a file
    # Here we have the two columns
    DATA = np.genfromtxt(Name,usecols=np.arange(0,2),max_rows=nxny+1)
    Dat=DATA[1:,:] # Remove the first raw with the header
    # Get U component and reshape as grid
    V_X=Dat[:,0]; V_X_g=np.reshape(V_X,(n_y,n_x))
    # Get V component and reshape as grid
    V_Y=Dat[:,1]; V_Y_g=np.reshape(V_Y,(n_y,n_x))
    # Get velocity magnitude
    V_MAGN=np.sqrt(V_X_g**2+V_Y_g**2)
    # Sample the three probes
    P1[k]=V_MAGN[i1,j1]; P2[k]=V_MAGN[i2,j2]; P3[k]=V_MAGN[i3,j3];
    print('Loading Step '+str(k+1)+'/'+str(n_t))

Figure 2 shows the signals in P1 (on the left) and P3 (on the right), together with their histogram and their power spectral densities. The meaning of these quantities will be evident by the end of the lecture. For the moment, it suffices to notice that this test case is characterized by a large scale variation of the free-stream velocity.

Figure 2. Velocity time series in probes P1 (on the left) and P3 (on the right) for the TR-PIV dataset analyzed in this lecture. See Figure 1 for the probe location.

This is evident from the velocity evolution of both probes, but it is clearer in P1 because this probe is sufficiently far from the cylinder wake to capture the evolution of the free-stream flow. In the first 1.5s, the free stream velocity is at approximately U∞≈12 m/sU_\infty\approx12\,\mathrm{m/s}. During this time, the flow is stationary (more about this in the next session).

Between t=1.5st=1.5\mathrm{s} and t=2.5st=2.5\mathrm{s} the velocity drops down to U∞≈8 m/sU_\infty\approx8\,\mathrm{m/s}, reaching a second stationary phase that last until the end of the acquisition. The variation of the flow velocity is sufficiently low to let the vortex shedding adapt and hence preserve an approximately constant Strouhal number of St=fd/U∞≈0.19St=fd/U_\infty\approx0.19, with d=5mmd=5\mathrm{mm} the diameter of the cylinder. Consequently, the vortex shedding varies from f≈459Hzf\approx459\mathrm{Hz} to f≈303Hzf \approx303\mathrm{Hz}. The two peaks in the PSD in probes P3 clearly shows both frequencies, although a frequency analysis does not allow to identify when these occur. This will also be briefly addressed later. The same peaks are much weaker but still present in P1.

In the histogram of both probes, two distributions are visible. These corresponds to the first (tk<1t_k < 1) and the second (tk>2.5t_k > 2.5) stationary conditions. The next section is dedicated to statistical analysis time series like these.

Statistics

This section presents some basic concepts and tools to study time series. We introduce the notion of random processes and random variables in the subsection 3.1, along with the relevant first and second-order statistics. As a first exercise, we study the ensemble statistics of an Ornstein Uhlenbeck process before recalling the notion of stationarity in the strong and weak sense.

The subsection 3.2 introduce the notion of ergodicity and the time statistics for finite duration signals. These are linked, in 3.3, to fundamental statistical quantities characterizing turbulent flows, further discussed in the second exercise.

Finally, 3.4 completes the introduction by reviewing the spectral characterization of stochastic signals. The review thus includes the notion of power spectral density and coherency explored in the third exercise.

The material covered falls within well-known textbooks in statistical inference and signal processing. Excellent references with a focus on deterministic signals are Oppenheim et al. [46]; Ljung and Glad [29]; Hsu [22]; Hayes [19], while Smith [64] provides a comprehensive treatment of Fourier analysis and filters. A gentle introduction to stochastic signals is provided by Kay [26]; Williamson [74]; Oppenheim [45] while Wang [72] provides an excellent review of orthogonal transforms.

First and Second Order Statistics

Each of the nx=71×ny=30n_x = 71 \times n_y = 30 grid points in the TR-PIV measurements generates a time series of nt=13200n_t = 13200 equally spaced samples in time. We will use the notation un(tk)u_n(t_k) to refer to the time series generated at the n−thn-th spatial location, with n∈[0,nxny]n \in[0,n_x n_y] and tk=kΔtt_k = k\Delta t the discrete time axis. Denoting as x=(xi,yj)\mathbf{x} = (x_i,y_j) the Cartesian grid from which the PIV data is available, with i∈[0,ny−1]i \in[0,n_y - 1] and j∈[0,nx−1]j \in[0,n_x - 1], we can see nn as a matrix linear index scanning all the points of the grid. Accordingly, we use xn\mathbf{x}_n to denote the location of the probe.

Unless otherwise stated, the time series unu_n under analysis is the velocity magnitude at that location1^{1} xn\mathbf{x}_n:

un(tk)=u(xn,tk)2+v(xn,tk)2.u_n(t_k) = \sqrt{u(\mathbf{x}_n,t_k)^2 + v(\mathbf{x}_n,t_k)^2}.

In a statistical treatment of signals, we admit that it is not possible (or simply not convenient) to describe signals with a deterministic model: we will not be able (nor we want to attempt) to predict the exact value un(tk)u_n(t_k) observed at a certain time. Instead, we will be content with the ability to attribute a probability to un(tk)u_n(t_k).

More concretely, we treat a time series as the realization of a random process, and we treat the value that the time series takes at a specific time as a random variable. A random process can be seen as a family of jointly distributed random variables indexed by the time tkt_k. Let U1,U2,…UntU_1,U_2,\ldots U_{n_t} denote the set of random variables associated to the value of un(t1),un(t2),…un(tnt)u_n(t_1),u_n(t_2),\ldots u_n(t_{n_t}). A full probabilistic characterization requires the joint probability density function of ntn_t random variables:

fU1,U2,…Unt(un(t1),un(t2),…un(tnt)).(1)f_{U_1,U_2,\ldots U_{n_t}}\left(u_n(t_1),u_n(t_2),\ldots u_n(t_{n_t})\right). \tag*{(1)}

This is a ntn_t-order description, and it is clearly impractical in most cases of interest. It is also not particularly interesting because we are usually not interested in computing the probability that a specific realization of ntn_t samples occurs.

On the other hand, we are always interested in a first order description. In other words, we seek to predict the probability of observing a certain un(tk)u_n(t_k) at a certain time tkt_k. This is a ’one-point statistics’, governed by the univariate probability density function (pdf) fU(un(tk))f_U(u_n(t_k)) associated to the random variable UnU_n. This pdf allows to assign the probability that um≤un(tk)<uMu_m \le u_n(t_k) < u_M:

P{um≤un(tk)<uM}=∫umuMfUk(Uk) dUk.(2)P\{u_m \le u_n(t_k) < u_M\} = \int_{u_m}^{u_M} f_{U_k}(U_k)\,\mathrm{d}U_k . \tag*{(2)}

Given this distribution, the mean (expectation) of the time series at tkt_k is:

μUn(tk)=E∼E[un(tk)]=⟨un(tk)⟩E=∫−∞∞UkfUk(Uk) dUk,(3)\mu_{U_n}(t_k) = \mathbb{E}_{\sim E}[u_n(t_k)] = \langle u_n(t_k)\rangle_E = \int_{-\infty}^{\infty} U_k f_{U_k}(U_k)\,\mathrm{d}U_k , \tag*{(3)}

having introduced the expectation operator E\mathbb{E} and the short hand notation with brackets ⟨⟩\langle\rangle. Similarly, the variance var⁡(un(tk))=σUk2(tk)\operatorname{var}(u_n(t_k)) = \sigma^2_{U_k}(t_k) of the time series at time tkt_k is:

σUk2(tk)=E∼E[(un(tk)−μUn(tk))2]=∫−∞∞(Uk−μUn(tk))2fUk(Uk) dUk,(4)\sigma^2_{U_k}(t_k) = \mathbb{E}_{\sim E}[(u_n(t_k)-\mu_{U_n}(t_k))^2] = \int_{-\infty}^{\infty}(U_k-\mu_{U_n}(t_k))^2 f_{U_k}(U_k)\,\mathrm{d}U_k , \tag*{(4)}

where σUk(tk)=var⁡(un(tk))\sigma_{U_k}(t_k) = \sqrt{\operatorname{var}(u_n(t_k))} is the standard deviation, also referred to as the root mean square of un(tk)u_n(t_k).

In a second order description of the time series, we might be interested in the statistical links between the value that the time series takes at time tkt_k and the values it takes at time tjt_j. The expectation of the product un(tk)un(tj)u_n(t_k)u_n(t_j) is the all important autocorrelation function, whose definition requires the (bivariate) joint probability function fUk,Uj(Uk,Uj)f_{U_k,U_j}(U_k,U_j):

RUU(tk,tj)=⟨un(tk)un(tj)⟩E=∫−∞∞∫−∞∞UkUjfUk,Uj(Uk,Uj) dUkdUj.(5)R_{UU}(t_k,t_j) = \langle u_n(t_k)u_n(t_j)\rangle_E = \int_{-\infty}^{\infty}\int_{-\infty}^{\infty} U_kU_j f_{U_k,U_j}(U_k,U_j)\,\mathrm{d}U_k\mathrm{d}U_j . \tag*{(5)}

Similarly, one can define the autocovariance function as

CUU(tk,tj)=⟨[un(tk)−μUn(tk)][un(tj)−μUn(tj)]⟩E=RUU(tk,tj)−μUn(tk)μUn(tj).(6)C_{UU}(t_k,t_j) = \langle[u_n(t_k)-\mu_{U_n}(t_k)][u_n(t_j)-\mu_{U_n}(t_j)]\rangle_E = R_{UU}(t_k,t_j)-\mu_{U_n}(t_k)\mu_{U_n}(t_j) . \tag*{(6)}

It is easy to note that CUU(tk,tk)=σ2(tk)C_{UU}(t_k,t_k)=\sigma^2(t_k).

Finally, we might be interested in the statistical link between two different time series; for examples the ones generated at two different locations. Denoting these as un(tk)u_n(t_k) and ul(tj)u_l(t_j) we recall the celebrated cross-correlation (the engine of PIV analysis!), which requires the (bivariate) joint probability function fUk,Wj(Uk,Wj)f_{U_k,W_j}(U_k,W_j) where UkU_k is the random variable associated to un(tk)u_n(t_k) and WjW_j is the one associated to ul(tj)u_l(t_j). We have:

RUW(tk,tj)=⟨un(tk)ul(tj)⟩E=∫−∞∞∫−∞∞UkWjfUk,Wj(Uk,Wj) dUkdWj(7)R_{UW}(t_k,t_j) = \langle u_n(t_k)u_l(t_j)\rangle_E = \int_{-\infty}^{\infty}\int_{-\infty}^{\infty} U_kW_j f_{U_k,W_j}(U_k,W_j)\,\mathrm{d}U_k\mathrm{d}W_j \tag*{(7)}

and the cross-covariance is simply

CUW(tk,tj)=⟨[un(tk)−μUn(tk)][ul(tj)−μUl(tj)]⟩E=RUW(tk,tj)−μUn(tk)μUl(tj).(8)C_{UW}(t_k,t_j) = \langle[u_n(t_k)-\mu_{U_n}(t_k)][u_l(t_j)-\mu_{U_l}(t_j)]\rangle_E = R_{UW}(t_k,t_j)-\mu_{U_n}(t_k)\mu_{U_l}(t_j) . \tag*{(8)}

All the above quantities are difficult to evaluate because none of the invoked distributions is usually known. To compute these important statistics, one must then also estimate the underlying distributions, and the only way to do this is to sample many instances of the time series. The set of instances we will be working with is called ensamble and the expectation operator should now be generalized to the notion of ensamble averaging. Let us review this concept in some detail for the mean in (3); we leave the extensions of the others statistics as an exercise. Let us assume that a total of nrn_r realizations has been collected (each being of length ntn_t and indexed by tkt_k) and let un(tk;r)u_n(t_k;r) be the r−thr-th realization at time tkt_k.

Adopting a frequentist approach to probabilities, we can compute the probability that the value un(tk)u_n(t_k) is taken at tkt_k as fV(un(tk),tk)=nu/nrf_V(u_n(t_k),t_k)=n_u/n_r, where nun_u is the number of times such value has been observed at time tkt_k within the full ensemble3.

Then, replacing the integral with a summation leads to

μUn(tk)=⟨u(tk)⟩E(tk)=∑r=0nr−1un(tk;r)fUk(un(tk))=1nr∑j=0nrun(tk;r).(9)\mu_{U_n}(t_k)=\langle u(t_k)\rangle_E(t_k)=\sum_{r=0}^{n_r-1}u_n(t_k;r)f_{U_k}(u_n(t_k))=\frac{1}{n_r}\sum_{j=0}^{n_r}u_n(t_k;r). \tag*{(9)}

For reasons that will soon become obvious, it is convenient to construct a snapshot matrix Un∈Rnt×nrU_n\in\mathbb{R}^{n_t\times n_r} with every column collecting a realization of length ntn_t of the stochastic process in the probe located at nn, and every row containing the nrn_r realizations at time tkt_k. Every realization, at the moment, must be thought of as a different experiment. With this arrangement of the data, the ensemble average of the series is the average across the rows of UnU_n. This is equivalent to multiplying the matrix by a vector of length ntn_t containing all 1/nr1/n_r terms. Similarly, the ensemble variance at all times can be computed along all rows of UnU_n. The computation of both quantities is shown in the first exercise.

Moving to second order statistics, the computation of the autocorrelation function requires averaging all products un(tk)un(tj)u_n(t_k)u_n(t_j). The ensemble autocorrelation in the matrix formalism becomes4

RUU(tk,tj)=⟨un(tk),un(tj)⟩E=1nrUnk(Unj)T,(10)R_{UU}(t_k,t_j)=\langle u_n(t_k),u_n(t_j)\rangle_E=\frac{1}{n_r}U_n^k(U_n^j)^T, \tag*{(10)}

where Unk∈R1×nrU_n^k\in\mathbb{R}^{1\times n_r} is the row vector collecting all the realizations un(tk;r)u_n(t_k;r) at time tkt_k, the superscript TT denotes transposition. This is essentially an inner product. The autocovariance is simply CUU(tk,tj)=RUU(tk,tj)−μU(tk)μU(tj)C_{UU}(t_k,t_j)=R_{UU}(t_k,t_j)-\mu_U(t_k)\mu_U(t_j), and the reader should notice that CUU(tk,tk)=σU2(tk)C_{UU}(t_k,t_k)=\sigma_U^2(t_k).

The cross correlation between two variables is analogous to (10): denoting as Un∈Rnt×nrU_n\in\mathbb{R}^{n_t\times n_r} and Wn∈Rnt×nrW_n\in\mathbb{R}^{n_t\times n_r} the snapshot matrices collecting the sets of nrn_r ensembles for the two random processes, the cross correlation reads

RUW(tk,tj)=⟨un(tk),wn(tj)⟩E=1nrUnk(Wnj)T.(11)R_{UW}(t_k,t_j)=\langle u_n(t_k),w_n(t_j)\rangle_E=\frac{1}{n_r}U_n^k(W_n^j)^T. \tag*{(11)}

Finally, it is worth noticing that in many fields, the definitions of CUUC_{UU}, CUWC_{UW} are normalized so that their value is in the range [−1,1][-1,1]. The normalized quantities are distinguished with a hat and read:

μX=∑j=13xjpj=252+154+253.\mu_X=\sum_{j=1}^{3}x_jp_j=\frac{2}{5}2+\frac{1}{5}4+\frac{2}{5}3.

This is no different than the simple arithmetic average in (9).

C^UU(tj,tk)=CUU(tj,tk)/σU2(tk)C^UW(tj,tk)=CUW(tj,tk)/(σU(tj)σW(tk)).(12)\widehat{C}_{UU}(t_j,t_k)=C_{UU}(t_j,t_k)/\sigma_U^2(t_k) \qquad\widehat{C}_{UW}(t_j,t_k)=C_{UW}(t_j,t_k)/(\sigma_U(t_j)\sigma_W(t_k)). \tag*{(12)}

As a note of warning, the reader should note that in a vast portion of the literature in experimental fluid mechanics, statistics are computed on the ‘fluctuating’ component of the velocity fields, following a classic Reynolds decomposition (Pope, [51]) (see also § 4.7). Therefore, the notion of autocovariance and autocorrelation are used interchangeably.

It is now worth pausing and practice with an exercise.

Exercise 1: The statistics of an Ornstein-Uhlenbeck Process

Consider the Ornstein-Uhlenbeck process governed by the following stochastic differential equation (see Øksendal [43]; Borodin and Salminen [10]; Pope [51])

dXt=κ(θ−Xt)dt+σdWt(13)dX_t=\kappa(\theta-X_t)dt+\sigma dW_t \tag*{(13)}

where the subscript tt here refers to a time step, κ>0\kappa>0 is called rate of reversion and controls how quickly the process reaches its (stationary) long-term behaviour, characterized by mean θ\theta and random fluctuation with standard deviation σ\sigma. The second term WtW_t is a stochastic term, and it is here taken as a normal Gaussian with zero mean and unitary standard deviation, here written as N(0,1)\mathcal{N}(0,1). The first term is often referred to as drift, the second as diffusion.

Consider a case with initial condition X0=0X_0=0, κ=1.2\kappa=1.2, θ=2\theta=2 and σ=1\sigma=1. Consider nt=1001n_t=1001 samples, with a sampling of Δt=0.01\Delta t=0.01 s. This corresponds to a physical observation time of T=10T=10s. Plot four realizations to explore the process.

The questions are two. (1) compute the ensemble mean, the ensemble standard deviation and the ensemble autocovariance as a function of time using nr=100n_r=100. (2) Study the convergence of these statistical quantities at two time steps (say at k=10k=10 and k=700k=700) as a function of the number of samples in the ensemble.

Solution. Let us begin by creating a function that produces a sample of the process, taking as input the four process parameters κ\kappa, θ\theta, σ\sigma and vector of times tkt_k. Using a simple loop, the following function does the job (see Python exercise 1):

1  import numpy as np
2  # Function definition
3  def U_O_Process(kappa,theta,sigma,t):
4      n_T=len(t)
5      # Initialize the output
6      y=np.zeros(n_T)
7      # Define Drift and Diffusion functions in the process
8      drift= lambda y,t: kappa*(theta-y)
9      diff= lambda y,t: sigma
10     noise=np.random.normal(loc=0,scale=1,size=n_T)*np.sqrt(dt)
11     # Solve Stochastic Difference Equation
12     for i in range(1,n_T):
13         y[i]=y[i-1]+drift(y[i-1],i*dt)*dt+diff(y[i-1],i*dt)*noise[i]
14     return y

The following script creates nr=500n_r=500 realizations and store them in a matrix UnU_n

of size nr×nrn_r \times n_r. Then, it plots four randomly chosen samples, namely the numbers r=1,10,22,55r=1,10,22,55. The four realizations are shown in Figure 3.1, with the plot axis being customized (see the provided codes).

import matplotlib.pyplot as plt
# Initial and final time
t_0=0; t_end=10
# Number of Samples
n_t=1001
# Process Parameters
kappa=1.2; theta=3; sigma=0.5
# Create the time scale
t=np.linspace(t_0,t_end,n_t); dt=t[2]-t[1]
# Collect 500 sample and store in U_N
n_r=500; U_N=np.zeros((n_t,n_r))
for l in range(n_r):
    U_N[:,l]=U_O_Process(kappa,theta,sigma,t)
    # Plot the results
    plt.figure(1)
    plt.plot(t,U_N[:,1])
    plt.plot(t,U_N[:,10])
    plt.plot(t,U_N[:,22])
    plt.plot(t,U_N[:,55])

The reader should play with the code long enough to realize that this process is characterized by a transient time of the order of 0.3s within which all signals move from zero to a stationary conditions (a precise definition of stationary signals is yet to come). Having arranged the data into ‘snapshot matrices’, the computation of the time average and the temporal standard deviation can be done in one line each:

U_Mean=np.mean(U_N,axis=1) # Ensamble Mean
U_STD=np.std(U_N,axis=1) # Ensamble STD

These are essentially ‘row-wise’ statistics.

Figure 3.1 shows the time average together with the range μU(tk)±σU(tk)\mu_U(t_k) \pm\sigma_U(t_k). It appears that the standard deviation grows gently until it reaches a constant value after about tk>4t_k>4s.

Finally, we analyze the ensamble autocorrelation of this random process. The ensamble cross-correlation between the time steps tjt_j and tkt_k for the ensambles Un\mathbf{U}_n and Wn\mathbf{W}_n of two random variables (see (11)) can be combuted with the following function:

def Norm_Autocov_0(U_N,W_N,k,j):
    n_r,n_t=np.shape(U_N)
    # Select all realizations at time t_k for U
    U_N_k=np.expand_dims(U_N[k,:],axis=0);
    # Select all realizations at time t_kj for W
    W_N_k=np.expand_dims(W_N[j,:],axis=0)
    # Note (These are row vectors)
    # Compute the average products
    PR=U_N_k.T.dot(W_N_k)
    R_UW=np.mean(PR)/(np.std(U_N_k)*np.std(W_N_k))
    return R_UW
Four randomly chosen samples and ensemble average of the Ornstein–Uhlenbeck process

Figure 3. Fig (a): Four randomly chosen samples of the Ornstein-Uhlenbeck process; Fig (b): Ensamble average of the process (in blue), along with the μ+σ\mu+ \sigma and μ−σ\mu- \sigma curves.

Having added a normalization. Then, the following script computes the autocorrelation of 100 randomly pairs, separated by a lag of 50 samples:

# Define lag (in number of samples)
lag=50
# Study the autocorrelation between two points at equal lags
N_S=100; R_UW=np.zeros(N_S)
# Select a 100 random points i (larger than 500)
J=np.random.randint(500,800,N_S); K=J+50
for n in range(N_S):
    R_UW[n]=Norm_Autocov_0(U_N,U_N,J[n],K[n])

The resulting set of autocorrelations has a mean of 0.545 and a standard deviation of 0.0059: in other words, it does not matter what the exact pair j,kj,k is, as long as they differ by the same lag (in this case, 50). This is because here we are sampling in time intervals from 500 to 800, and here the process has reached its stationary condition. The reader is encouraged to repeat the exercise at an earlier interval.

Finally, we analyze the convergence of the statistics as a function of the number of realizations. The following script computes the mean and the standard deviation at k=10k=10 and k=700k=700 for a number of realizations that goes from 1 to 1000.

n_R=np.round(np.logspace(0.1,3,num=41))
# Prepare the outputs at k=100
mu_10=np.zeros(len(n_R))
sigma_10=np.zeros(len(n_R))
# Prepare the outputs at k=700
mu_700=np.zeros(len(n_R))
sigma_700=np.zeros(len(n_R))
# Loop over all n_R’s.
for n in range(len(n_R)):
    # show progress
    print('Computing n='+str(n)+' of '+str(len(n_R)))
    n_r=int(n_R[n]) # Define the number of ensambles
    U_N=np.zeros((n_t,n_r)) # Initialize the ensamble set
    for l in range(n_r): # Fill the Ensamble Matrix
        U_N[:,l]=U_O_Process(kappa,theta,sigma,t)
# Compute the mean and the std's
mu_10[n]=np.mean(U_N[10,:]) # Ensamble Mean
sigma_10[n]=np.std(U_N[10,:]) # Ensamble STD
mu_700[n]=np.mean(U_N[700,:]) # Ensamble Mean
sigma_700[n]=np.std(U_N[700,:]) # Ensamble STD

By analyzing the vector nRn_R in the script, the reader should note that some of the entries with a small number of samples are taken multiple times to show the variability in the prediction. The results are shown in Figure 4.

Four randomly chosen samples of the Ornstein–Uhlenbeck process and ensemble average statistics

Figure 4. Fig (a): Four randomly chosen samples of the Ornstein-Uhlenbeck process; Fig (b): Ensamble average of the process (in blue), along with the μ+σ\mu+ \sigma and μ−σ\mu- \sigma curves.

It can be shown that the convergence of both the mean and the standard deviation is ∝nr\propto\sqrt{n_r}, but the convergence of the mean is ∝σU\propto\sigma_U while the convergence of the standard deviation is ∝σU2\propto\sigma_U^2 (see also [see also 23]): this explains why a larger number of samples is needed to converge the second-order statistics, and why the convergence at k=100k = 100 is slower. The reader is encouraged to explore the provided scripts further, to observe this result in action at different points.

We close this section with the definition of stationarity. A process is stationary in a strict sense (SSS) if all its statistics are invariant to a shift of the origin. In other words, we could add any t0t_0 to the times t1,t2,…tntt_1,t_2,\ldots t_{n_t} in (1) and still have the same distribution. A less restrictive concept is that of a process which is stationary in a weak sense (WSS). In this case, only the first and second order statistics are time invariant. As a result, the ensamble mean is constant, i.e. μUn(tk)=E[un(tk)]=μU∈R\mu_{U_n}(t_k) = \mathbb{E}[u_n(t_k)] = \mu_U \in\mathbb{R}, and the autocorrelation/autocovariance are only function of the time delay between the two points considered: RUU(tk,tk+τ)=RUU(τ)R_{UU}(t_k,t_k+\tau) = R_{UU}(\tau) and CUU(tk,tk+τ)=CUU(τ) ∀tkC_{UU}(t_k,t_k+\tau) = C_{UU}(\tau)\ \forall t_k.

Ergodicity and Finite Durations

Ensamble averaging requires repeating experiments hundreds of times. This is not feasible. However, because stationary signals’ statistics are time-invariant, it is reasonable to wonder whether a sufficiently long experiment can be statistically equivalent to many short ones.

This brings us to the notion of ergodicity. This is a sophisticated concept (see also [30, 45, 11], but the essential idea can be explained as follows: a process is ergodic if ensemble statistics can be replaced by temporal statistics on any particular realization5. We avoid introducing the statistics in the continuous domain and move straight to the statistics for a discrete sequence like the ones encountered in practice. The time average is defined as

μ~U=⟨un(tk)⟩T=lim⁡nt→∞1nt∑k=0nt−1un(tk),(14)\tilde{\mu}_U = \langle u_n(t_k)\rangle_T = \lim_{n_t\to\infty} \frac{1}{n_t}\sum_{k=0}^{n_t-1}u_n(t_k), \tag*{(14)}

while the temporal autocorrelation is

R~UU(τl)=⟨un(tk),un(tk+τl)⟩T=lim⁡nt→∞12nt−1∑k=−nt+1nt−1un(tk)un(tk+τl).(15)\tilde{R}_{UU}(\tau_l) = \langle u_n(t_k),u_n(t_k+\tau_l)\rangle_T = \lim_{n_t\to\infty}\frac{1}{2n_t-1}\sum_{k=-n_t+1}^{n_t-1}u_n(t_k)u_n(t_k+\tau_l). \tag*{(15)}

The expectation is now taken in time (i.e. ET{}\mathbb{E}_T\{\}). A subscript TT is used to distinguish from the expectation in the ensemble set, and a tilde is used to distinguish the temporal statistics from the ensemble ones. While the definition of the time average is rather obvious, the one of autocorrelation deserves a deeper discussion.

Both quantities converge at sufficiently large ntn_t for a stationary signal while none of them is meaningful for non-stationary signals. We come back to the convergence problem with an exercise at the end of this section. For the moment, the main concern in the reader’s mind should be a practical one: even assuming that ntn_t is large enough (hence the limit can be dropped), eq (15) cannot be computed for finite duration signals unless additional information is provided.

Let us illustrate this with an example: compute the autocorrelation of the time series u(tk)=[1,2,3,4]u(t_k) = [1, 2, 3, 4]. Consider a Python-like notation, hence u(t0)=0u(t_0) = 0 and u(t3)=4u(t_3) = 4. Considering tk=kΔtt_k = k\Delta t, we could use an index notation u[k]u[k] and write u[0]=0u[0] = 0 or u[3]=4u[3] = 4 regardless of the Δt\Delta t. Because this signal has nt=4n_t = 4 entries, we have a total of 2nt−1=72n_t - 1 = 7 possible lags ll. The lags in (15) take the form τl=lΔt\tau_l = l\Delta t, and the autocorrelation will thus be a vector of length 7 which can also be indexed as R~UU(τl)=R~UU[l]\tilde{R}_{UU}(\tau_l) = \tilde{R}_{UU}[l] with l∈[−3,3]l \in[-3, 3]. Figure 5 illustrate the problem of computing R~UU[−2]\tilde{R}_{UU}[-2]: this is the correlation between the signal u[k]u[k] and its copy shifted by 2 entries to the right u[k−2]u[k-2].

Sketch showing linear and circular correlation with zero and cyclic padding

Figure 5. Sketch of the difference between linear and cyclic autocorrelation. The figure shows how these two definitions compute the autocorrelation entry R~UU[−2]\widetilde{R}_{UU}[-2] of the vector u[k]=[1,2,3,4]u[k] = [1, 2, 3, 4]. The linear correlation pads with zero on both sides while the cyclic correlation assumes periodicity with period ntn_t.

While it is clear that the summation will contain 3×1+4×23 \times1 + 4 \times2, it is unclear how to deal with the entries that do not have a match in the other signal because of the finite size of the signal. On a practical level, the two classic solutions are zero padding and cyclic padding.

The zero-padding leads to the linear correlation: the vectors are padded by zeros in all the entries lacking information. The cyclic padding leads to the cyclic correlation: the vectors are assumed periodic with period ntn_t. We distinguish these operators with and LL or a CC, and the results for this example is:

R~UUL[−2]=11/7R~UUC[−2]=22/7.(16)\tilde{R}_{UUL}[-2] = 11/7 \quad\tilde{R}_{UUC}[-2] = 22/7. \tag*{(16)}

Ergodicity and Finite Durations

Note that RUUCR_{UUC} is periodic and for both operators we have R~UU[l]=R~UU[−l]\widetilde{R}_{UU}[l] = \widetilde{R}_{UU}[-l]. This is why one often plots only6 the last ntn_t vectors of the autocorrelation, corresponding to the lags from l=0l = 0 to l=nt−1l = n_t - 1. The same considerations are valid for the computation of the autocovariance, the cross correlations and the cross covariance, and the reader should be able to derive their formulae.

When the time series are ‘short’, R~UUL\widetilde{R}_{UUL} and R~UUC\widetilde{R}_{UUC} are very different, and it is essential to clarify which is being used. ‘Short’ here means that relevant lags (giving normalized cross-correlation of ∼1\sim1) are present within a time scale that is comparable to the duration of the signal. This is the case of PIV interrogation (see Käufer (2020)), in which time is replaced by space, and the discrete-time indices are the shifts in pixels.

The linear (acyclic) operators are unbiased estimators (Kay, [26]) and provide better statistical convergence to their ensemble counterparts. On the other hand, the cyclic operators are computationally more interesting because the periodic assumption enables a link with the cyclic convolution, which can be computed in the frequency domain using the Fast Fourier Transform (FFT). Briefly, it is easy to note that the cyclic convolution between two vectors x[k]x[k] and y[k]y[k] differs from the cyclic correlation by the flipping of the second vector (see Hayes [19]). In the frequency domain, this flipping corresponds to the conjugation of the associated Fourier transform.

To be more specific, let X(fn)X(f_n) be the Discrete Fourier Transform (DFT) of x[n]x[n], i.e.:

X(fn)=F{x[k]}=1nt∑k=0nt−1x[k]e−2πfnkΔt↔x[k]=F−1{X[k]}=∑k=0nt−1X(fn)e2πfnkΔt(17)X(f_n) = \mathcal{F}\{x[k]\} = \frac{1}{n_t}\sum_{k=0}^{n_t-1}x[k]e^{-2\pi f_n k\Delta t} \leftrightarrow x[k] = \mathcal{F}^{-1}\{X[k]\} = \sum_{k=0}^{n_t-1}X(f_n)e^{2\pi f_n k\Delta t} \tag*{(17)}

where fn=nΔff_n = n\Delta f, with n=0,nt−1n = 0, n_t - 1 and Δf=fs/nt\Delta f = f_s/n_t the frequency resolution, with fs=1/Δtf_s = 1/\Delta t the sampling frequency. The cyclic cross-correlation between two signals x[k]x[k], y[k]y[k], having DFT X(fn)X(f_n), Y(fn)Y(f_n) can be computed as

RXY=F−1(F[x[k]]F[y[k]]‾)=F−1(X(fn)Y(fn)‾),(18)R_{XY} = \mathcal{F}^{-1}\left(\mathcal{F}[x[k]]\overline{\mathcal{F}[y[k]]}\right) = \mathcal{F}^{-1}\left(X(f_n)\overline{Y(f_n)}\right), \tag*{(18)}

with the overbar denoting complex conjugation. A Python (and normalized) implementation of the cross covariance is thus provided by following function:

def R_UW_C(u,v):
    RUU=np.fft.ifft(np.abs(np.fft.fft(u)*np.fft.fft(v))).real
    c=(RUU/len(u)-(np.mean(u)*np.mean(v)))/(np.std(u)*np.std(v))
    return c[:len(u)//2]

The auto-covariance is thus simply R_UW_C(u,u)R\_UW\_C(u,u) while moving from covariances to correlations is only a matter of removing or keeping the time average.

It is possible to compute the linear operators using a circular extension if an appropriate zero padding is used before the cyclic extension (see Smith [64]). This is what software packages like Scipy and Numpy do when computing the ’FFT-based’ cross-correlation. The linear (and normalized) cross-covariance using the Scipy’s function correlate is implemented in the following function:

from scipy import signal
def R_UW_L(u,v):
    # Call to the scipy function correlate:
    RUU = signal.correlate(u-np.mean(u), v-np.mean(v),\
    mode=’same’,method=’auto’)/len(u)/(np.std(u)*np.std(v))
    return RUU[RUU.size//2:]

The entry ‘method’, set by default to ‘auto’, uses the fastest option between direct and FFT-based correlation, depending on the size of the array. The reader will need these functions for the next exercise. But first, we might need to recall some more definitions.

A note on Turbulence

The statistical treatment of turbulence is a vast subject, far beyond the scope of this lecture. The reader is referred to Pope [51]; Tennekes [67]; Davidson [13]; Mcdonough [32] for an comprehensive introduction on the topic and to Saarenrinne and Piirto [55]; Lavoie et al. [28]; Segalini et al. [60]; Scharnowski et al. [56]; Ayegba and Edomwonyi-Otu [4]; Wang et al. [71] for a discussion on the impact of measurement resolution on the main turbulence variables. This section recalls the definitions that are required to solve the provided exercises.

In the classic Reynolds decomposition of stationary turbulent flows, the velocity field is decomposed into the sum of an average and a fluctuating part. For the provided 2D dataset, we write these as:

u(xi,tk)=⟨u⟩(xi)+u′(xi,tk),(19)\mathbf{u}(\mathbf{x}_i,t_k) = \langle\mathbf{u}\rangle(\mathbf{x}_i) + \mathbf{u}'(\mathbf{x}_i,t_k), \tag*{(19)}

where xi\mathbf{x}_i denotes a Cartesian grid, ⟨u⟩=(⟨u⟩,⟨v⟩)T\langle\mathbf{u}\rangle= (\langle u\rangle,\langle v\rangle)^T is the mean-field and u′=(u′,v′)T\mathbf{u}' = (u',v')^T is the fluctuating field. The averaging in this formulation is an ensemble averaging. However, under the assumption of stationarity and ergodicity, this can be safely replaced by a more convenient time averaging.

The intensity of the fluctuation part is measured in terms of standard deviation or root mean square velocity, with components urms=σU(xi)u_{rms} = \sigma_U(\mathbf{x}_i), vrms=σV(xi)v_{rms} = \sigma_V(\mathbf{x}_i). A second-order

A note on Turbulence

description is provided by the time correlation matrix of the fluctuating components7, which is a 2D flow reads:

⟨ui′,uj′⟩=(⟨u′2⟩⟨u′v′⟩⟨v′u′⟩⟨v′2⟩)=(urms2R~U,V(0)R~V,U(0)vrms2).(20)\left\langle u_i',u_j'\right\rangle= \begin{pmatrix} \left\langle u'^2\right\rangle& \left\langle u'v'\right\rangle\\ \left\langle v'u'\right\rangle& \left\langle v'^2\right\rangle \end{pmatrix} = \begin{pmatrix} u_{\mathrm{rms}}^2 & \widetilde{R}_{U,V}(0) \\ \widetilde{R}_{V,U}(0) & v_{\mathrm{rms}}^2 \end{pmatrix}. \tag*{(20)}

This matrix varies at every location and in a statistical treatment of turbulence is linked to the Reynolds stress tensor τi,j=−ρ⟨ui′,uj′⟩\tau_{i,j}=-\rho\left\langle u_i',u_j'\right\rangle, namely the additional stresses used to model the non-linear diffusion effect produced by turbulence on the mean flow. The trace of this matrix is linked to the turbulence kinetic energy κ=1/2(urms2+vrms2)\kappa=1/2(u_{\mathrm{rms}}^2+v_{\mathrm{rms}}^2), while the ratios TIu=urms/⟨u⟩TI_u=u_{\mathrm{rms}}/\langle u\rangle and TIv=vrms/⟨v⟩TI_v=v_{\mathrm{rms}}/\langle v\rangle define the turbulence intensity vector TI=(TIu,TIv)TI=(TI_u,TI_v).

The Reynolds stress tensors provides information on the degree of anisotropy of turbulence: in isotropic turbulence, one has ⟨u,v⟩=0\langle u,v\rangle=0 and ⟨u2⟩=⟨v2⟩\langle u^2\rangle=\langle v^2\rangle; hence the matrix in (20) is diagonal. The eigenvalues of this matrix can be used to study the principal direction of turbulence momentum exchange [16, 61] using graphical representation such as the popular Lumley’s triangle [31]. This is essentially a principal component decomposition of the velocity components in a point.

Similarly, the covariance between gradients of the fluctuating components is linked to the turbulent dissipation rates ε\varepsilon. In its complete 3D form, this is defined as ε=ν⟨∂xiuj(∂xjui+∂xiuj)⟩\varepsilon=\nu\langle\partial_{x_i}u_j(\partial_{x_j}u_i+\partial_{x_i}u_j)\rangle, with ν\nu the kinematic viscosity, having used the indices i,j=1,2,3i,j=1,2,3 to index the velocity components and axes and having implied summation. The full expression counts 12 terms and it is of difficult evaluation in a 2D measurement because of the missing information in the direction normal to the measurement plane (see Saarenrinne and Piirto [55]; Wang et al. [71] for an interesting discussion). This quantity measures the rate at which turbulence energy is converted into thermal energy by viscous dissipation and is essential in the modeling of turbulent flows. Because of the strong simplifications required by its computation, and because of the presence of gradients (which need extra care!), the exercise does not ask to compute this quantity8.

Finally, an important statistical quantity is the integral time scale Θ\Theta. For the fluctuating component uu, having normalized temporal autocorrelation R~UU/σU\widetilde{R}_{UU}/\sigma_U, this is defined (see also Oliveira et al. [44]) as

Θ=∫0∞R~UU(τ)σU dτ.(21)\Theta=\int_{0}^{\infty}\frac{\widetilde{R}_{UU}(\tau)}{\sigma_U}\,d\tau. \tag*{(21)}

This integral converges (and the definition makes sense) only if the autocorrelation function tends to zero sufficiently fast. When this occurs, the integral time scale measures the time after which the random process becomes uncorrelated with itself or, in a more pictorial interpretation, the time within which the variable ‘remembers’ its history.

The reader is now ready for the second exercise.

Exercise 2: The statistics of a turbulent Flow

Study the statistics of the provided dataset, focusing on tk≤1t_k \le1. (1) Compute the mean-field and the turbulence intensity field. (2) Analyze the convergence of these quantities in probe P1 and probe P2. (3) Compute the time autocorrelations in P1 and P2. If meaningful, compute the integral time scale. Test if the use of linear or cyclic operators makes any difference. Finally, (4) compute the Reynolds stress tensor at both locations.

Solution. We begin with a minor modification to code listing 1, where the velocity magnitude was sampled in the provided probes. We now store the full snapshots into a snapshot matrix U containing all fields of uu and a snapshot matrix V containing all fields of vv. This matrix will be important for the second part of the lecture. We also extract both velocity components in the probes P1,P2,P3. Here’s how the loop should be modified (see Exercise_2.py):

# Initialize the probes
P1_U=np.zeros(n_t,dtype="float32")
P1_V=np.zeros(n_t,dtype="float32")
P2_U=np.zeros(n_t,dtype="float32")
P2_V=np.zeros(n_t,dtype="float32")
P3_U=np.zeros(n_t,dtype="float32")
P3_V=np.zeros(n_t,dtype="float32")
# Prepare the snapshot matrices U and V
D_U=np.zeros((n_s//2,n_t),dtype="float32")
D_V=np.zeros((n_s//2,n_t),dtype="float32")
for k in range(0,n_t):
    # Name of the file to read
    Name=FOLDER+os.sep+'Res%05d'%(k+1)+'.dat'
    # Read data from a file
    # Here we have the two columns
    DATA = np.genfromtxt(Name,usecols=np.arange(0,2),\
    max_rows=nxny+1)
    Dat=DATA[1:,:] # Remove the first raw with the header
    # Get U component and reshape as grid
    V_X=Dat[:,0]; V_X_g=np.reshape(V_X,(n_y,n_x))
    # Get V component and reshape as grid
    V_Y=Dat[:,1]; V_Y_g=np.reshape(V_Y,(n_y,n_x))
    # Assign the vector like snapshots to snapshot matrices
    D_U[:,k]=V_X; D_V[:,k]=V_Y
    # Sample the U components
    P1_U[k]=V_X_g[i1,j1]; P2_U[k]=V_X_g[i2,j2]; P3_U[k]=V_X_g[i3,j3];
    # Sample the V Components
    P1_V[k]=V_Y_g[i1,j1]; P2_V[k]=V_Y_g[i2,j2]; P3_V[k]=V_Y_g[i3,j3];
    print('Loading Step '+str(k+1)+'/'+str(n_t))

Both the snapshot matrices and the velocity components sampled in probes are saved in a npz file for the exercises in the second part. Moreover, to limit memory usage, we work with variables in float32 rather than float64; this is accurate enough for our purposes and let the exercise comply with the limited memory of the author’s laptop.

Using the snapshot matrices, the mean flow and the root mean square velocity components can be computed in one line each:

A note on Turbulence

# U Component
mu_U=np.mean(D_U,axis=1) # Ensemble Mean
sigma_U=np.std(D_U,axis=1) # Ensemble STD
# V component
mu_V=np.mean(D_V,axis=1) # Ensemble Mean
sigma_V=np.std(D_V,axis=1) # Ensemble STD

Plotting these fields is now only a matter of reshaping the results back onto a matrix and using the provided functions (see the python code). The mean flow and the turbulence intensity TI=TIu2+TIv2TI=\sqrt{TI_u^2+TI_v^2} are shown in Figure 6. Note that the latter is computed by normalizing the root mean square component with respect to the free stream velocity and not the local mean velocity. Most of the turbulence is localized in the wake region, where the fluctuation amplitude reaches up to 50% of the free stream velocity. This answers question (1).

Mean flow and turbulence intensity

Figure 6. Fig (a): Mean flow of the provided dataset in the first 1s (3000 snapshots) (b): Turbulence intensity, normalizing the root mean square field by the free stream.

The analysis of the convergence of the statistics at the probe locations P1 and P3 is similar to what was performed in the previous exercise. This is left as an exercise for the reader. The results are shown in Figure 7. The convergence in P1 is particularly fast, as the root mean square velocity is low and the mean velocity is large. The same is not true in P3, where convergence is barely achieved. The reader should analyze these statistics alongside the histogram in Figure 2: the tk≤1t_k \leq1s of acquisition is associated with the distribution centred at approximately 12 m/s.

Convergence of statistics at probe locations P1 and P3

Figure 7.

The reader is referred to Bruun [11] (chapter 12) for a discussion on the convergence of the mean and standard deviation as a function of the turbulence intensity.

Figure 7: Fig (a): Mean flow of the provided dataset in the first 1s1s (3000 snapshots) (b): Turbulence intensity, normalizing the root mean square field by the free stream.

For the scope of this lecture, we can declare point (2) answered, although the reader is encouraged to proceed with further analysis9.

The time autocorrelations in P1 and P2 can be computed with the functions provided in the previous sections. Because the time series under analysis are rather long (3000 samples), the difference between circular and linear autocorrelation is not appreciable for lags τl<0.1s\tau_l < 0.1s; the reader is nevertheless encouraged to explore the difference at larger lags. The results for P1 and P2, using circular autocorrelation, are shown in Figure 3.3 for the first 0.01s0.01s. In both probes, oscillations of the flow are visible, although these are much stronger in P3 than in P1. As a result, none of the autocorrelations operators converge to zero, meaning that the signals are not purely ‘stochastics’ and a deterministic component could (and should!) be extracted before an integral time scale is to be computed10. Interestingly, the oscillations appear in phase, as if the entire flow field is locked at a given frequency.

Figure 8. Fig (a): Mean flow of the provided dataset in the first 1s1s (3000 snapshots) (b): Turbulence intensity, normalizing the root mean square field by the free stream.

Finally, we close with the Reynolds stress tensor. This is essentially a covariance matrix. Arranging the velocity information in matrices of dimention 3000×23000 \times2, this is simply:

1  u1_f = P1_U - np.mean(P1_U);    v1_f = P1_V - np.mean(P1_V)
2  U_f_1 = np.vstack((u1_f, v1_f)).T
3  Re_1 = 1/len(P1_U)*U_f_1.T.dot(U_f_1)

The results in points P1 and P2 are:

⟨ui,uj⟩(P1)=(0.0254−0.0022−0.00220.0169)⟨ui,uj⟩(P3)=(0.3337−0.0068−0.00680.2793)(22)\langle u_i,u_j\rangle(P1)=\begin{pmatrix}0.0254 & -0.0022 \\ -0.0022 & 0.0169\end{pmatrix} \qquad \langle u_i,u_j\rangle(P3)=\begin{pmatrix}0.3337 & -0.0068 \\ -0.0068 & 0.2793\end{pmatrix} \tag*{(22)}

The scatter plot of the fluctuation components are shown in Figure 9. It is clear that the turbulent kinetic energy and the turbulent intensity is much larger in P3 than P1, and hence the distribution is broader in the first. In both points, however, the scatter plot shows a rather uncorrelated distribution of fluctuation, resulting in a nearly diagonal covariance: the flow is almost isotropic. ‘Almost’, because urmsu_{rms} is slightly larger than vrmsv_{rms} in both points.

Scatter plots of the fluctuating components at probes P1 and P3

Figure 9. Scatter plot of the fluctuating components v−⟨v⟩v-\langle v\rangle and u−⟨u⟩u-\langle u\rangle at the location of probe P1 (Fig a) and probe P2 (Fig b).

Power Spectral Density and Coherence

We now move to the frequency representation of stochastic signals. This is slightly more involved than the frequency representation of deterministic signals. In a continuous domain, a stochastic signal is seldomly square-integrable, and it thus seldomly admits an ordinary Fourier transform. In the signal processing terminology, the definition of an appropriate frequency domain requires shifting the treatment from the notion of energy to the notion of power11: stochastic signals have generally infinite energy but finite power (Oppenheim, [45]; Hayes [19]).

E{x[k]}=lim⁡nt→∞∑k=0nt∣x[k]∣2P{x[k]}=lim⁡nt→∞12nt−1∑k=0nt∣x[k]∣2.(23)\mathcal{E}\{x[k]\}=\lim_{n_t\to\infty}\sum_{k=0}^{n_t}|x[k]|^2 \qquad \mathcal{P}\{x[k]\}=\lim_{n_t\to\infty}\frac{1}{2n_t-1}\sum_{k=0}^{n_t}|x[k]|^2. \tag*{(23)}

Therefore, we shall not focus on the Fourier transform of a signal but at the Fourier transform of its autocorrelation (which is square integrable). The following transform exist for all signals of interest:

E{un2(tk)}=R~UU(0)=12π∫−∞∞SUU(ω)dω,(24)\mathbb{E}\{u_n^2(t_k)\}=\widetilde{R}_{UU}(0)=\frac{1}{2\pi}\int_{-\infty}^{\infty}S_{UU}(\omega)d\omega, \tag*{(24)}

where ω=f/2π\omega=f/2\pi is the pulsation and ff is the frequency and SUUS_{UU} is the continuous Fourier transform of the autocorrelation function and is known as power spectral density. Note that this is real and even in ω\omega because the time autocorrelation RUUR_{UU} is symmetric (RUU(τ)=RUU(−τ)R_{UU}(\tau)=R_{UU}(-\tau)). The Fourier pairs of interest are thus

SUU(ω)=∫−∞∞R~UU(τ)e−jωτdτandR~UU(τ)=12π∫−∞∞SUU(ω)ejωτdω.(25)S_{UU}(\omega)=\int_{-\infty}^{\infty}\widetilde{R}_{UU}(\tau)e^{-j\omega\tau}d\tau \quad\text{and}\quad \widetilde{R}_{UU}(\tau)=\frac{1}{2\pi}\int_{-\infty}^{\infty}S_{UU}(\omega)e^{j\omega\tau}d\omega. \tag*{(25)}

These are also known as Wiener-Khinchin relations. The notion of power spectral density is important because it allows to give a spectral representation to stochastic signals and thus to generalize the theory of linear time-invariant (LTI) systems (see Mendez [35]). This theory brings powerful and simple tools for system identification, filtering and forecasting, and we show some of them in practice in the next exercise (see Oppenheim [45]; Smith [63] for more).

Concretely, we are interested in the notion of coherency in relation to the frequency content of signals. The question motivating this section is simple: is there any chance to predict what is happening in one probe (e.g. P2), at least within a certain range of frequencies, given the signal we have in another probe (e.g. P3)?

We first need some definitions. Let us assume that a stochastic signal x[k]∈Rntx[k]\in\mathbb{R}^{n_t} is the input of a linear and deterministic system which responds with a second stochastic signal y[k]∈Rnty[k]\in\mathbb{R}^{n_t}. Uncorrelated noise might be added to the output of this system and we only see the resulting yn[k]=y[k]+N[k]y_n[k]=y[k]+N[k]. If the power of the noise is too large, we might not be able to recover any reasonable estimate of y[k]y[k] and we will say that there is a poor level of coherency between x[k]x[k] and yn[k]y_n[k]. Conversely, we might be able to identify an approximation of the underlying linear linking input and output. This link is here to be analyzed frequency by frequency.

Considering finite duration signals, the convergence problems of the Fourier representation are less stringent, and we can waive some of the formalism required for the continuous world: under the assumption of circular extension of the signals12, every digital signal has a Discrete Fourier Transform (DFT). Let X(fn)X(f_n) and Y(fn)Y(f_n) denote the DFT of the input x[k]x[k] and the output y[k]y[k] (see (17) for the definitions).

If a linear time invariant system links y[k]y[k] to x[k]x[k], the output can be computed via convolution of the input with the system’s impulse response. From the convolution theorem, we know that in frequency domain this is a multiplication with the transfer function of the system H(fn)H(f_n), i.e. the DFT of the impulse response. We thus have:

y[k]=∑m=0nt−1x[k] h[k−m] ⟷ Y(fn)=H(fn)X(fn).(26)y[k]=\sum_{m=0}^{n_t-1}x[k]\,h[k-m]\ \longleftrightarrow\ Y(f_n)=H(f_n)X(f_n). \tag*{(26)}

We now introduce the discrete equivalent of (25). It is possible to show (see Oppenheim et al. [46] and (18)) that the power spectral density of the input is

SXX(fn)=∑k=0nt−1RXX[k]e−2πfnkΔt=F{F−1{X(fn)X(fn)‾}}=X(fn)X(fn)‾.(27)S_{XX}(f_n)=\sum_{k=0}^{n_t-1}R_{XX}[k]e^{-2\pi f_n k\Delta t}=\mathcal{F}\{\mathcal{F}^{-1}\{X(f_n)\overline{X(f_n)}\}\}=X(f_n)\overline{X(f_n)}. \tag*{(27)}

Similarly, the power spectral density of the output is SYY(fn)=Y(fn)Y(fn)‾S_{YY}(f_n)=Y(f_n)\overline{Y(f_n)} and we can additionally define the cross-spectral density as SYX=X(fn)Y(fn)‾S_{YX}=X(f_n)\overline{Y(f_n)}.

We can now craft a spectral function which measures how well the spectrum of the output correlates with the spectrum of the input. This is the coherence function:

CYX(fn)=∣SYX(fn)∣2SXX(fn)SYY(fn).(28)C_{YX}(f_n)=\frac{\lvert S_{YX}(f_n)\rvert^2}{S_{XX}(f_n)S_{YY}(f_n)}. \tag*{(28)}

At those frequencies for which Y(fn)=H(fn)X(fn)Y(f_n)=H(f_n)X(f_n) (i.e. for which a LTI system could model the input-output relation), one has

SYX(fn)=Y(fn)X(fn)‾=H(fn)X(fn)X(fn)‾→SYX(fn)=H(fn)SXX(fn).S_{YX}(f_n)=Y(f_n)\overline{X(f_n)}=H(f_n)X(f_n)\overline{X(f_n)}\to S_{YX}(f_n)=H(f_n)S_{XX}(f_n).

Moreover, noticing that SYY(fn)=∣H(fn)∣2SXX(fn)S_{YY}(f_n)=\lvert H(f_n)\rvert^2S_{XX}(f_n), we recover CYX(fn)=1C_{YX}(f_n)=1. At frequencies for which no LTI system can link the two spectra, the coherence is much close to zero. The function is undefined at fnf_n’s for which either SXXS_{XX} or SYYS_{YY} is zero.

To conclude this section, it is worth noticing that working with one single spectrum for both input and outputs makes little sense: an ensemble of time series leads to an ensemble of spectra. The classic approach to evaluating a stochastic signal’s frequency representation involves averaging, which could be either in the ensemble domain or in the time domain. Under the assumption of ergodicity, the second is usually preferred, and the result is the well known Welch’s method [73] or periodogram method. The computation is performed by dividing the signal into successive (and overlapping) blocks, computing the DFT of these, and then averaging the results. In practice, a smoothing window w[k]w[k] multiplies the signal in the time domain to cope with the problems arising by the (usually violated) assumption of periodicity. For SXX(fn)S_{XX}(f_n), for instance, we have:

SXX(fn)=1nL∑m=1nL−1∣XWm(fn)∣2 with XWm(fn)=F{x[k]w[k]}(29)S_{XX}(f_n)=\frac{1}{n_L}\sum_{m=1}^{n_L-1}\lvert XW_m(f_n)\rvert^2\text{ with }XW_m(f_n)=\mathcal{F}\{x[k]w[k]\} \tag*{(29)}

Here XWm(fn)XW_m(f_n) is the DFT of the windowed block xm[k]w[k]x_m[k]w[k], with xm[k]=x[k+mnW]x_m[k]=x[k+mn_W], wm[k]w_m[k] the window function designed to taper gracefully to zero at both endpoints of the block, and k=0,1,…nw−1k=0,1,\ldots n_w-1 with nLn_L denoting the number of blocks. Because the multiplication in the time domain is a convolution in the frequency domain, the method is essentially a smoothing operation on the signal’s spectrum; this is why the method is sometimes also called ‘smoothed spectrum’ and implemented in the frequency domain.

The Python’s scipy package offers robust functions to compute both the power spectral densities and the spectral coherence. We use both in the next exercise.

Exercise 3: Power Spectral Densities and Coherency The power spectral density of the signal in the probes P1 and P3 was shown in Figure 2. As in the previous exercise, let us consider only the first tk≤1 st_k\leq1\ \mathrm{s} of

acquisition. First, practice with the notion of coherency by testing it on a synthetic signal yy. To construct this signal, filter the signal in P2 with a moving average filter of order three, i.e., impulse response h=[1,1,1]/3h = [1,1,1]/3, and add noise to the result. Next, show the original and synthetic signal’s power spectral densities and study the coherence function between the two signals. (1) What do you get and why? Finally, (2) study the coherency between the signals in P2 and P3.

Solution. We assign the signal to the input variable xx and perform the filtering plus noise superposition in the following lines:

1  data = np.load(’Sampled_PROBES.npz’)
2  n_T=3000; Fs=3000; dt=1/Fs
3  P2_U=data[’P2_U’][0:n_T];
4  P3_U=data[’P3_U’][0:n_T];
5  # Prepare the time axis
6  t=np.linspace(0,dt*(n_T-1),n_T)
7  # Define impulse response and ’input’
8  h=np.ones(3)/3; x=P2_U
9  # We padd to use the frequency domain:
10 h=np.concatenate([h,np.zeros(len(x)-3)])
11 # The transfer function will be
12 H=np.fft.fft(h)
13 # This is circular convolution
14 X=np.fft.fft(x)
15 # Add noise
16 N=np.random.normal(0,0.5*np.std(x),len(x))
17 y=np.real(np.fft.ifft(X*H))+N

We compute the power spectral densities SXXS_{XX} and SYYS_{YY} as follows:

1  N_P=300
2  f,Sxx_den = signal.welch(x, Fs, nperseg=N_P, scaling=’density’)
3  f,Syy_den = signal.welch(y, Fs, nperseg=N_P, scaling=’density’)

The reader should consult the documentation of the welch function. This function performs mean removal by default, while the scaling option ’density’ makes the output a density function (i.e. units are ‘V²/Hz’, with V the input units). The blocks for the averaging are here shown to be of size 300 while the overlapping is set by default to 50%. The power spectral densities of the original and the manipulated signal is shown in Figure 10a). The spectra are quite similar at low frequencies and feature the same peak at 459Hz. However, they become increasingly different at higher frequencies.

Power spectral densities and cross-coherency plots

Figure 10. Fig (a): Power spectral densities of the original signal (stream-wise velocity component in P1) together with its manipulated version (b):cross-coherency between the signal and its manipulated version, together with the magnitude of the transfer function used in the manipulation.

We now compute the coherence function together with the transfer function of the moving average filter here implemented and plotting the results in the same graph:

1  # Prepare the transfer function of h
2  # curiosity: this is a nice tool for transf functions
3  wz, Hz = signal.freqz(h)
4  # Create the figure axis
5  fig, ax = plt.subplots(figsize=(6, 3))
6  # Compute the coherency
7  f, Cxy = signal.coherence(x, y, Fs, nperseg=N_P)
# Plot the results in decibels
plt.plot(f, 20*np.log10(Cxy),label='$C_{x,y}$')
# Plot the transfer function
plt.plot(wz/np.pi*Fs/2,20*np.log10(Hz), 'r',label='$|H(f)|$')

The result is in Figure 10b). It is clear that the coherency decreases following the same trend as the transfer function of the LTI system implemented: frequencies that are almost removed from the original signal leave the stage to the added random (incoherent) noise. The reader should repeat the exercise without adding noise.

Finally, the cross coherency between P2 and P3 is shown in Figure 3.4. The overall level of coherence is low, except for the regions at fn≈459f_n \approx459Hz. This corresponds to the frequency of the vortex shedding, characterized by coherent structures. We look for these in the next section.

Figure 11. Cross-coherency between the stream-wise velocity in P2 and P3. At approximately

Classic Modal Analysis

In the last exercise, we observed a strong correlation between the signals in P2 and P3 at the frequency of the vortex shedding. In the previous section, we have considered time series arising by sampling a specific point (and we took care of including a subscript nn in all the defined quantities).

In this section, we look for the statistics, the coherence and the frequency content of the flow field as a whole: the time series of interest is no longer a scalar, but a vector of dimension ns=nxnyn_s = n_x n_y for the components uu and vv, and dimension 2ns2n_s if we take both components in the same ‘snapshot’.

More specifically, let u⃗(xi,tk)=(u(xi,t),v(xi,t))\vec{u}(x_i,t_k) = (u(x_i,t),v(x_i,t)) denote the TR-PIV field over a uniform grid xi∈Rnx×nyx_i \in\mathbb{R}^{n_x \times n_y}, with ii a matrix linear index, and a temporal discretization tk={(k−1)Δt}k=1k=ntt_k = \{(k-1)\Delta t\}_{k=1}^{k=n_t} with Δt=1/fs\Delta t = 1/f_s the time step and fsf_s the sampling frequency. In what follows, bold fonts are used to indicate matrices.

The goal of modal analysis is to describe the data as a linear combination of elementary information referred to as modes. Each mode has its own spatial structure and temporal evolution and can potentially describe essential features in the data. Each mode has a spatial structure ϕ⃗k(xi)\vec{\phi}_k(x_i), a temporal structure ψr(tk)\psi_r(t_k), and an amplitude σr\sigma_r that weights its importance. The sampled velocity field is thus expanded as

u⃗(xi,tk)=∑r=1RC⃗r(xi) ψr(tk)=∑r=1Rσr ϕ⃗r(xi) ψr(tk),(30)\vec{u}(x_i,t_k) = \sum_{r=1}^{R} \vec{C}_r(x_i)\,\psi_r(t_k) = \sum_{r=1}^{R} \sigma_r\,\vec{\phi}_r(x_i)\,\psi_r(t_k), \tag*{(30)}

where the contribution of each mode is defined as σr=∥C⃗r(xi)∥2\sigma_r = \lVert\vec{C}_r(x_i) \rVert_2 and the spatial structures are obtained via normalization ϕ⃗r=C⃗r(xi)/σr\vec{\phi}_r = \vec{C}_r(x_i)/\sigma_r.

Assuming that all modes are (at least!) linearly independent, the number (RR) of modes non-null amplitudes σr\sigma_r defines the dimensionality of the data for a given basis. Truncating the summation to R~<R\widetilde{R} < R modes provides an approximation of order R~\widetilde{R} with respect to a specific decomposition.

These tools were initially developed for identifying coherent structures in turbulent flows ([30, 8, 62]) and to construct reduced-order models ([21, 20, 7]) of fluid flows. Their range of application is nowadays much broader and include (among others) filtering ([53]), image processing ([33]) and validation of numerical simulations ([17]). By removing ‘irrelevant’ modes, we use the decomposition as a filter, and we compress data. By focusing on the dynamics described by ‘a few’ modes, we use decompositions as tools for constructing reduced-order models, which can significantly reduce the computational burdens in the simulations of large dynamical systems. A recent review of dimensionality reduction is proposed by Ahmed et al. [2].

In this section, we consider three decompositions in the form of eq. (30). These are the Discrete Fourier Transform (DFT) in subsection 4.1, the Proper Orthogonal Decomposition (POD) in subsection 4.2 and the Dynamic Mode Decomposition (DMD) in subsection 4.3. These can be distinguished by the choice of the temporal structure ψr(tk)\psi_r(t_k) (see also Mendez [34]). We will not go into much implementation and algorithmic details (these are extensively treated in Mendez [34]) and we will use the Python package MODULO (see Ninni and Mendez [41])

In what follows, we use the subscripts F\mathcal{F}, D\mathcal{D}, and P\mathcal{P} to denote, respectively, the modes linked to the DFT, the DMD and the POD.

The DFT

The DFT Modes

In the DFT, the temporal basis ψFr(tk)\psi_{Fr}(t_k) is the Fourier basis, which we have already encountered in (17): that is ψFr(tk)=1/ntexp⁡(2πfrtk)\psi_{Fr}(t_k)=1/\sqrt{n_t}\exp(2\pi f_r t_k) where fr=rΔff_r=r\Delta f, with r=0,…,nt−1r=0,\ldots,n_t-1, Δf=fs/nt\Delta f=f_s/n_t the frequency resolution and fs=1/Δtf_s=1/\Delta t the sampling frequency.

The only difference with respect to the previous section (and the classic implementations in Python or Matlab) is that we will now normalize the DFT by 1/nt1/\sqrt{n_t} and not 1/nt1/n_t. This normalization makes the basis ψFr(tk)\psi_{Fr}(t_k) orthonormal: if we construct a vector ψFr[n]=1/ntexp⁡(2πfrkΔt)\psi_{Fr}[n]=1/\sqrt{n_t}\exp(2\pi f_r k\Delta t), for any of the frequencies frf_r we will have ∥ψr∥2=1\|\psi_r\|_2=1.

We now compute the DFT at every location and arrange the results in the form of spatial modes. From a practical point of view, we can leverage the organization of the data into a snapshot matrix and compute the DFT at every row. Again, this will be very convenient and make the decomposition accessible in just one line of code.

Let us consider the DFT of the signal at one specific location xi\mathbf{x}_i:

F{u⃗(xi,tk)}(xi,fr)=1nt∑k=0nt−1u⃗(xi,tk)e−2πfrtk=C⃗Fr(xi).(31)\mathcal{F}\{\vec{u}(x_i,t_k)\}(x_i,f_r)=\frac{1}{\sqrt{n_t}}\sum_{k=0}^{n_t-1}\vec{u}(x_i,t_k)e^{-2\pi f_r t_k}=\vec{C}_{Fr}(x_i). \tag*{(31)}

This operation can be interpreted in at least two ways. From a statistical point of view, this is a correlation (in time) between the temporal evolution of the signal at a certain position u⃗(xi,tk)\vec{u}(x_i,t_k) and a certain basis element ψFr(tk)\psi_{Fr}(t_k). From a linear algebra point of view, this is a projection (or ‘inner product’) of the temporal evolution of the signal on to the basis element ψFr(tk)\psi_{Fr}(t_k) (see Mendez [35]).

The result of this operation is a set of coefficients C⃗r(xi)\vec{C}_r(x_i) (for the uu and the vv components) which is in essense a velocity field for every frequency frf_r. This field represents the Fourier spectra of the time series of uu and the vv at that specific point xi\mathbf{x}_i and pulsating at the single frequency frf_r. For each of these fields, we can compute the norm (the space domain) as σFr=∥C⃗Fr(xi)∥2\sigma_{Fr}=\|\vec{C}_{Fr}(x_i)\|_2 and the normalized result will be denoted as ϕ⃗Fr=C⃗Fr(xi)/σFr\vec{\phi}_{Fr}=\vec{C}_{Fr}(x_i)/\sigma_{Fr}. The set of ϕ⃗Fr\vec{\phi}_{Fr} gives the spatial structure (basis) of the DFT modes.

The decomposition derived can be arranged as a matrix factorization of the snapshot matrix D=[d1,d2,…dnt]∈Rns×nt\mathbf{D}=[\mathbf{d}_1,\mathbf{d}_2,\ldots\mathbf{d}_{n_t}]\in\mathbb{R}^{n_s\times n_t}, in which each column dk\mathbf{d}_k contains the snapshot u⃗(xi,tk)\vec{u}(x_i,t_k) properly reshaped. For the DFT, this will be:

D=ΦFΣFΨF,(32)\mathbf{D}=\mathbf{\Phi}_F\mathbf{\Sigma}_F\mathbf{\Psi}_F, \tag*{(32)}

where ΦF=[ϕ⃗F1,ϕ⃗F2,…ϕ⃗Fnt]∈Rns×nt\mathbf{\Phi}_F=[\vec{\phi}_{F1},\vec{\phi}_{F2},\ldots\vec{\phi}_{Fn_t}]\in\mathbb{R}^{n_s\times n_t} collects the spatial structures along its columns, ΣF\mathbf{\Sigma}_F is a diagonal matrix collecting the spectra σFr\sigma_{Fr} and ΨF=[ψF1,ψF2,…ψFnt]∈Rnt×nt\mathbf{\Psi}_F=[\psi_{F1},\psi_{F2},\ldots\psi_{Fn_t}]\in\mathbb{R}^{n_t\times n_t} is the Fourier matrix collecting the temporal structures (note that this matrix is symmetric, i.e. ΨF=ΨFT\mathbf{\Psi}_F=\mathbf{\Psi}_F^{\mathsf{T}}). Let us compute these for the provided dataset.

Exercise 4: The DFT Modes of the TR-PIV data

Consider the dataset in the first tk≤1t_k\leq1s. Using the fft\mathit{fft} algorithm, compute the DFT modes. Plot the spectra σFr\sigma_{Fr} and the spatial structure of the modes with fr≈450Hzf_r\approx450Hz, namely the frequency of the vortex shedding.

Solution. We have the dataset already arranged in the form of snapshot matrices from the previous exercise. We load these and stack into a single snapshot matrix, to which we remove the time average:

# Load the snapshot matrices from Exercise 2
data = np.load(’Snapshot_Matrices.npz’)
Xg=data[’Xg’]; Yg=data[’Yg’]
D_U=data[’D_U’]; D_V=data[’D_V’]
# Assembly both components in one single matrix
D_M=np.concatenate([D_U,D_V],axis=0) # Reshape and assign
# We remove the time average (for plotting purposes)
D_MEAN=np.mean(D_M,1) # Temporal average (along the columns)
D=D_M-np.array([D_MEAN,]*n_t).transpose() # Mean Removed

The computation of the DFT fields C⃗Fr\vec{C}_{F_r} can be done in one line of code:

PHI_SIGMA = (np.fft.fft(D, n_t, 1)) / (n_t ** 0.5)

Note that this is the product ΦFΣF\Phi_F\Sigma_F in (32) (hence the name). The remaining steps are (1) a normalization of all the columns, storing the results of the amplitudes in a vector:

# Initialize the PHI_F MATRIX (spatial structures)
PHI_F = np.zeros((D.shape[0], n_t), dtype=complex)
# Initialize the SIGMA_F vector (DFT Spectra)
SIGMA_F = np.zeros(n_t)
# Normalization loop
for r in tqdm(range(0, n_t)):
    SIGMA_F[r] = abs(np.vdot(PHI_SIGMA[:, r], PHI_SIGMA[:, r]))** 0.5
    PHI_F[:, r] = PHI_SIGMA[:, r] / SIGMA_F[r]

and (2) the sorting of the modes in decreasing order of amplitude:

# find indices for sorting in decreasing order
Indices = np.flipud(np.argsort(SIGMA_F))
# Sort all the sigmas
Sorted_Sigmas = SIGMA_F[Indices]
# Sort all the frequencies accordingly.
Sorted_Freqs = Freqs[Indices]
# Sorted Spatial Structures Matrix
Phi_F = PHI_F[:, Indices]
# Sorted Amplitude Matrix
SIGMA_F = np.diag(Sorted_Sigmas)

where the vector ‘Freqs’ is the one computed using scipy’s function

Freqs = np.fft.fftfreq(n_t) * Fs # Compute the frequency bins

This is essentially fr=rΔff_r=r\Delta f defined in the previous section. The resulting spectra is plotted in Figure 12a) versus the associated frequencies and in Figure 12b) in terms of the mode index rr. The first plot shows that, as expected, the leading modes are the ones at the frequency of the vortex shedding. The second plot shows that the ‘decay rate’ of the amplitude is modest, and many modes are needed to approximate this dataset despite its strong oscillatory nature. We conclude that despite the periodicity in the data, the convergence of the DFT is not particularly strong.

Amplitudes of the DFT modes as a function of frequency and mode index

Figure 12. Fig (a): Amplitudes of the DFT modes σFr\sigma_{F_r} as a function of the associated frequency frf_r. The leading modes are close to the vortex shedding frequency. Fig (b): amplitude of the DFT modes σFr/σF1\sigma_{F_r}/\sigma_{F_1}, in decreasing order, as a function of the index rr.

The spatial structures can then be shown after a reshaping. The leading structure (see entry Sorted_Freqs[0] has f0=450f_0=450 Hz) and the reader should note that modes come in pairs (only the postive portion of the spectra is here shown, from fr=0f_r=0 to fr=fs/2f_r=f_s/2). The real and the imaginary part of the leading DFT modes can be reshaped as follows:

r =0;
# Take the real part of mode r
Phi_F_Mode_U_r = np.real(Phi_F[0:nxny,r]).reshape((n_x,n_y))
Phi_F_Mode_V_r = np.real(Phi_F[nxny::,r]).reshape((n_x,n_y))
# Take the imaginary part of mode r
Phi_F_Mode_U_i = np.imag(Phi_F[0:nxny,r]).reshape((n_x,n_y))
Phi_F_Mode_V_i = np.imag(Phi_F[nxny::,r]).reshape((n_x,n_y))

These fields are shown in Figure 13. Their superposition describes the travelling of vortices whose size is comparable to the cylinder. The two modes provide a simple kinematic approximation of the vortex shedding mechanism and explain why the signals in probe P2 and P3 have a strong level of coherency at fn≈450f_n \approx450 Hz: both probes are located in regions where the first mode is ‘strongly present’.

Real (a) and imaginary (b) parts of the spatial structure of the leading DFT mode

Figure 13. Real (a) and imaginary (b) parts of the spatial structure of the leading DFT mode, capturing the vortex shedding.

While this exercise nicely showcased the strengths of the DFT; a simple variant can quickly showcase its limitations: the reader is invited to repeat the same exercise taking now all the nt=13000n_t=13000 snapshots of the dataset. The convergence decreases significantly, and two dominant frequencies will now appear (linked to the vortex shedding in the two stationary conditions, as described in § 3). These will have their associated modes, but the DFT will leave an important question open: when are these modes active?

An extension of tools from time-frequency analysis (e.g. Wavelets, [15]) to the ‘modal’ formalism could address these points but does not always improve the convergence and keeps the decomposition ’general’, i.e. with bases not adapted to the dataset at hand. The POD introduced in the next session gives the best possible convergence and tailors its basis to the data.

The POD

The previous exercise has highlighted the frequency localization capabilities of the DFT, but also its (generally) poor convergence. The DFT does not qualify as a ‘data-driven’ decomposition because the basis employed is pre-defined: in the DFT, we use the Fourier basis regardless of the data at hand.

We now enter into data-driven decompositions and thus seek to tailor the basis to the data. The Proper Orthogonal Decomposition (POD) is the oldest technique and arguably the most robust and simple tool. The decomposition is known with different names in different fields, and the reader is referred to Dawson [14] for a review.

The POD is the fundamental ‘energy-based’ decomposition: it provides the modes with the highest amplitude, and such that the convergence of an approximation with increasing number of modes is optimal. Identifying the temporal structures satisfying this condition is a constrained optimization problem (see (see Holmes et al. (1996) or Bishop (2006)) in the framework of Principal Component Analysis, PCA) and we here give the solution without proofs: the POD’s temporal structures ψPr\psi_{Pr} must be eigenvectors of the temporal correlation matrix K[k,n]=⟨u~(xi,tk),u~(xi,tn)⟩SK[k,n] = \langle\widetilde{\mathbf{u}}(x_i,t_k),\widetilde{\mathbf{u}}(x_i,t_n)\rangle_S, with ⟨∙⟩S\langle\bullet\rangle_S the expectation operator in the space domain. Hence, we must have:

K ψPr(tk)=λr ψPr(tk)∀r∈[1,R]⟶K=∑r=1Rλr ψPr(tk)ψPrT(tk)(33)\mathbf{K}\,\psi_{Pr}(t_k) = \lambda_r\,\psi_{Pr}(t_k)\quad\forall r \in[1,R] \longrightarrow\mathbf{K} = \sum_{r=1}^{R}\lambda_r\,\psi_{Pr}(t_k)\psi_{Pr}^{T}(t_k) \tag*{(33)}

having considered each temporal structure as a column vector, and having recalled, in the last summation, that the correlation matrix is symmetric and positive definite. The ψPr\psi_{Pr} are, therefore, orthonormal like the ψFr\psi_{Fr} in the DFT. What gives the POD the special status of ‘proper’ orthogonality, however, is that this specific choice of the temporal structure also makes the associated spacial structures ϕPr\phi_{Pr} orthonormal.

In addition to optimal convergence, this ‘proper orthogonality’ brings two interesting by-products (see also [37, 34]. The first is that the amplitudes of the modes are σPr=λr\sigma_{Pr} = \sqrt{\lambda_r} and thus the normalization step in the calculation of the spatial structures in (30) is no longer necessary. Then, it is possible to compute the spatial structures via correlation:

Φ⃗k(x⃗i)=1σr⟨u⃗(x⃗i,tk),ψPr(tk)⟩T=1σr∑r=1ntu⃗(x⃗i,tk)ψPr(tk).(34)\vec{\Phi}_k(\vec{x}_i)=\frac{1}{\sigma_r}\langle\vec{u}(\vec{x}_i,t_k),\psi_{Pr}(t_k)\rangle_T=\frac{1}{\sigma_r}\sum_{r=1}^{n_t}\vec{u}(\vec{x}_i,t_k)\psi_{Pr}(t_k). \tag*{(34)}

The second by-product is that the spatial structures are eigenvectors of the spatial correlation matrix C[i,j]=⟨u⃗(x⃗i,tk),u⃗(x⃗j,tk)⟩S\mathbf{C}[i,j]=\langle\vec{u}(\vec{x}_i,t_k),\vec{u}(\vec{x}_j,t_k)\rangle_S, with ⟨∙⟩S\langle\bullet\rangle_S the inner product in the time domain. Their orthogonality allows also to write:

ψPr(tk)=1σr⟨u⃗(x⃗i,tk),Φ⃗k(x⃗i)⟩S=1σr∑i=1nsu⃗(x⃗i,tk)Φ⃗k(x⃗i).(35)\psi_{Pr}(t_k)=\frac{1}{\sigma_r}\langle\vec{u}(\vec{x}_i,t_k),\vec{\Phi}_k(\vec{x}_i)\rangle_S=\frac{1}{\sigma_r}\sum_{i=1}^{n_s}\vec{u}(\vec{x}_i,t_k)\vec{\Phi}_k(\vec{x}_i). \tag*{(35)}

Therefore, one might decide to work with the diagonalization of K\mathbf{K} or C\mathbf{C} depending on whether one has nt≪nsn_t\ll n_s or ns≪1n_s\ll1, and compute by correlation the remaining structures (either the ϕPr\phi_{Pr}’s via (34) or the ψPr\psi_{Pr} via (35)).

Once completed, the arranging the POD in the matrix formalism as in (32) becomes:

D=ΦPΣPΨPT.(36)\mathbf{D}=\mathbf{\Phi}_P\mathbf{\Sigma}_P\mathbf{\Psi}_P^T. \tag*{(36)}

For datasets uniformly spaced in time and space, this is a Singular Value Decomposition (SVD) of D\mathbf{D}. We see this decomposition in action in the next exercise.

Exercise 5: The POD Modes of the TR-PIV data

Compute the POD of the full dataset (i.e. nt=13200n_t=13200, corresponding to tkt_k). Show the amplitude of the modes, their spatial structures and the frequency content of their temporal structures.

Solution. We should re-run the script from Exercise 2 and build the full dataset of nt=13200n_t=13200 snapshot. Assuming that this is stored in a matrix D\mathbf{D} as before, the first step is to compute the temporal correlation matrix. This is simply DTD\mathbf{D}^T\mathbf{D}. However, because the computational cost of this operation hits the limit of the author’s laptop, we here use the memory saving feature of the MODULO package ([41]), recently implemented in Python by Schena and Mendez [57]. The following code loads the data and computes the snapshot matrix:

# Load the snapshot matrices from Exercise 2
data = np.load('Snapshot_Matrices.npz')
Xg=data['Xg']; Yg=data['Yg']
D_U=data['D_U']; D_V=data['D_V']
# Assembly both components in one single matrix
D_M=np.concatenate([D_U,D_V], axis=0) # Reshape and assign
# We remove the time average (for plotting purposes)
D_MEAN=np.mean(D_M,1) # Temporal average (along the columns)
D=D_M-np.array([D_MEAN,]*n_t).transpose() # Mean Removed

To use the MODULO package, we first create a MODULO object with some important information, then run the POD computation in one line:

m = MODULO(data=D,
    n_Modes = n_r,
    MEMORY_SAVING=True,
    N_PARTITIONS=11)
# Prepare (partition) the dataset
D = m._data_processing() # Note that this will cancel D!
# Compute the mPOD
Phi_P, Psi_P, Sigma_P = m.compute_POD()

The first line in the script prepares the object declaring that the data to decompose is in the matrix DD, that we look for the first 500 modes, and that the memory saving feature will be requested with 11 partitions (Ninni and Mendez, [41]). This means that the code will break the dataset into 11 chunks and store it in temporary files, releasing some memory. Then, during the computation of the correlation matrix, the code loads and use one piece at a time. This allows computing the temporal correlation matrix saving considerable computational resources. The partitioning is created as soon as the second line in the script is run. Finally, the last line contains the computation of the POD.

The rest of the code is an exercise in reshaping and plotting data. Figure 4.2 shows the plot of the POD mode amplitude from the diagonal of Sigma_P. Interestingly, the amplitude of all modes tends to zero for r>3r > 3: the first three modes contains most of the information in the data.

Figure 14. Amplitude decay of the POD Modes for the full dataset.

As in the previous exercise, the spatial structures are first reshaped onto the PIV grid and then plotted as fields. Figure 15 collects the spatial structures and the spectra of the first three leading modes. These modes are extensively discussed in Mendez et al. [38]. Briefly, the first mode describes the large scale variation of the flow, while the other two represent the vortex shedding in its entire evolution. Because POD modes are real, travelling patterns can only be described by POD pairs whose temporal structures are in quadrature.

Spatial structures of the first three POD modes and spectra of the associated temporal structures

Figure 15. Spatial structures of the first three POD modes (a) and spectra of the associated temporal structures (b).

The compression achievable by the POD is interesting for many purposes. From a ‘dynamical system’ perspective, the dataset can be seen as a discrete system in Rns\mathbb{R}^{n_s}, with ns=4320n_s = 4320 spatial points (2130 per components). Predicting the flow field a time tk+1t_{k+1} requires predicting ns=4320n_s = 4320 numbers. If one approximates the dataset with respect to the (reduced) POD basis:

u⃗~(x⃗i,tk)=σP1ϕ⃗P1(x⃗i)ψP2(tk)+σP2ϕ⃗P1(x⃗i)ψP2(tk)+σP3ϕ⃗P3(x⃗i)ψP3(tk),(37)\tilde{\vec{u}}(\vec{x}_i,t_k) = \sigma_{P1}\vec{\phi}_{P1}(\vec{x}_i)\psi_{P2}(t_k) + \sigma_{P2}\vec{\phi}_{P1}(\vec{x}_i)\psi_{P2}(t_k) + \sigma_{P3}\vec{\phi}_{P3}(\vec{x}_i)\psi_{P3}(t_k), \tag*{(37)}

the flow field at time tk+1t_{k+1} is prescribed by 3 numbers, namely the value taken by the temporal structures ψP(tk)\psi_P(t_k). The three temporal coefficients evolve according to a dynamical system in R3\mathbb{R}^{3} and the spatial structures in (37) map (linearly!) this small dimensional system to the original one, producing an error which can be shown to be equal to σ4\sigma_{4} in a l2l_{2} sense [14, 35].

Figure 4.2 shows the trajectory of the reduced dynamical system in R3\mathbb{R}^{3}, which is simply the plot of the three temporal structures. In stationary conditions, the modes 2 and 3 describe a circular orbit whose diameter is larger at larger velocities. The transient phase moves the system state from the large to the smaller orbits along the axis of the first mode. At all times, the system stays on a paraboloid. The existence of this paraboloid has been derived analytically at low Reynolds numbers (Re=100Re = 100, see Noack et al. [42]); this exercise shows that this manifold continues to exists also at much higher ReRe and during transient states.

Figure 16. Evolution of the reduced order dynamical system in the POD basis.

The DMD

The DFT modes have one frequency each, but this frequency is constrained to be an integer multiple of the fundamental frequency f0=1/Tf_0 = 1/T with T=ntΔtT = n_t\Delta t the observation time. This potentially generates convergence issues due to windowing problems. The POD modes have the best possible convergence, but its modes are of difficult interpretation: all the information is extremely compressed into a few modes.

The Dynamic Mode Decomposition (DMD) solves potential windowing problems while putting the “frequency-based” formalism of the DFT into the “data-driven” or “tailored-basis” formalism of the POD. The idea (introduced by Schmid [59]) is to fit a linear dynamical system onto a reduced dynamical system derived via POD. A different formulation was proposed by Rowley et al. [54] and the reader is referred to Chen et al. [12] for an interesting discussion on the link between DMD and DFT.

The idea of fitting a linear dynamical system to the POD reduced-order model of the data has a long history, and it is interesting to note that modern DMD algorithms are remarkably similar to what was introduced in the late ’80s as Principal Oscillation Patterns (POP, Hasselmann [18]; von Storch and Xu [70]) or Linear Inverse Modeling (LIM, Penland [49]; Penland and Magorian [50]) in climatology (see also Tu et al. (2014); Mendez et al. [38]; Uruba [69]).

The basic algorithm described in this section is the one implemented in MODULO and proposed by Tu et al. (2014), which is briefly described in this section. Let D1=[d1,…,dnt−1]∈Rns×(nt−1)\mathbf{D}_1 = [\mathbf{d}_1,\ldots,\mathbf{d}_{n_t-1}] \in\mathbb{R}^{n_s \times(n_t-1)} and D2=[d2,…,dnt]∈Rns×(nt−1)\mathbf{D}_2 = [\mathbf{d}_2,\ldots,\mathbf{d}_{n_t}] \in\mathbb{R}^{n_s \times(n_t-1)} be two portions of the snapshot matrix differing by a one shift in time. If a linear dynamical system ’moves’ D1\mathbf{D}_1 to D2\mathbf{D}_2, then this system is characterized by a propagator matrix P∈Rns×ns\mathbf{P} \in\mathbb{R}^{n_s \times n_s} such that D2=PD1\mathbf{D}_2 = \mathbf{P}\mathbf{D}_1. The eigenvalues of this propagator governs the evolution of the system and its eigenvectors are the DMD modes we are looking for.

Solving for PP is a canonical least square problem, which could be solved via the pseudo-inverse of D1D_1. This can be computed via POD (i.e. SVD) of D1D_1:

P≈D2ΨPΣ~P−1ΦPT(38)P \approx D_2 \Psi_P \widetilde{\Sigma}_P^{-1} \Phi_P^T \tag*{(38)}

However, computing the eigendecomposition of PP is a poorly conditioned and computationally prohibitive problem in most real applications. The solution is then to leverage on the compression achievable by POD. Projecting the whole problem onto a reduced POD basis Φ~P=[ϕP1,…ϕPnr]\widetilde{\Phi}_P = [\phi_{P1}, \ldots\phi_{P n_r}], with nr≪ntn_r \ll n_t, the reduced system in Rnr×nr\mathbb{R}^{n_r \times n_r} becomes:

D2≈PD1⟶Φ~PTD2⏟V~2≈Φ~PTPΦ~P⏟S~Φ~PTD1⏟V~1⟶V~2≈S~V~1(39)D_2 \approx P D_1 \longrightarrow\underbrace{\widetilde{\Phi}_P^T D_2}_{\widetilde{V}_2} \approx\underbrace{\widetilde{\Phi}_P^T P \widetilde{\Phi}_P}_{\widetilde{S}} \underbrace{\widetilde{\Phi}_P^T D_1}_{\widetilde{V}_1} \longrightarrow\widetilde{V}_2 \approx\widetilde{S}\widetilde{V}_1 \tag*{(39)}

where the l2l_2 approximation lies in the second step, i.e. in the assumption that Φ~PΦ~PT≈I\widetilde{\Phi}_P\widetilde{\Phi}_P^T \approx I with II the identity matrix of size nr×nrn_r \times n_r.

In this reduced system, the reduced propagator is thus:

S~=Φ~PTPΦ~P=Φ~PTD2Ψ~PΣ~P−1(40)\widetilde{S} = \widetilde{\Phi}_P^T P \widetilde{\Phi}_P = \widetilde{\Phi}_P^T D_2 \widetilde{\Psi}_P \widetilde{\Sigma}_P^{-1} \tag*{(40)}

having introduced (38). The eigenvalue decomposition of this propagator S~=QΛQ−1\widetilde{S} = Q\Lambda Q^{-1} governs the evolution of the reduced system in (39). Structures linked to eigenvalues that have ∣λr∣<1|\lambda_r| < 1 vanish at large times while those that have ∣λr∣>1|\lambda_r| > 1 explode. Structures linked to ∣λr∣=1|\lambda_r| = 1 are purely harmonic, like the DFT modes. However, their frequency is not constrained to be multiple of the fundamental tone. Finally, the (complex) spatial structure of the DMD can be computed by back-propagating in the original space using the POD basis: ΦD=Φ~PQ\Phi_D = \widetilde{\Phi}_P Q.

Exercise 6: The DMD Modes of the TR-PIV data As discussed in Mendez et al. [38], the DMD of the full dataset encounters serious convergence problems. After all, the trajectories in 4.2 cannot be approximated by a linear dynamical system. We thus focus again on the dataset in the first tk≤1st_k \le1s (that is nt=3000n_t = 3000). Compute the DMD spectra and show the spatial structure of the dominant mode.

Solution. Using MODULO, the call to the DMD function is a one-line command, provided that the snapshot matrix has been prepared as in the previous exercises. The DMD with n_r=500 modes can be computed as follows:

#%% Perform the DMD with Modulo
# Initialize a ’MODULO Object’
m = MODULO(data=D,
MEMORY_SAVING=True,
n_Modes = n_r,
N_PARTITIONS=11)
# Prepare (partition) the dataset
D = m._data_processing() # Note that this will cancel D!
# Compute the POD
Phi_D, Lambda, freqs, a0s = m.compute_DMD()

The output provides the spatial structures of the DMD (in Phi_D), the eigenvalues of the reduced propagator (Lambda), the frequency (in Hz) linked to each eigenvalue

and the initial amplitudes of all modes (a0’s). We can now plot the DMD spectra in the complex plane:

1 fig, ax = plt.subplots(figsize=(4, 4)) # This creates the figure
2 plt.plot(np.imag(Lambda),np.real(Lambda),'ko')
3 circle = plt.Circle((0,0),1,color='b', fill=False)
4 ax.add_patch(circle)
5 ax.set_aspect('equal')

and in terms of frequencies:

1 fig, ax = plt.subplots(figsize=(6, 4)) # This creates the figure
2 plt.plot(freqs,np.abs(Lambda),'ko')

The results are shown in Figure 17a and b) respectively. We note that all the computed eigenvalues have absolute values smaller than one and will thus vanish in the long run. Two of these have ∣λ∣=0.9940742|\lambda| = 0.9940742 (see fig a): these are almost on the unitary circle in the complex domain and therefore are nearly harmonics. Interestingly, these correspond to fn=450.27 Hzf_n = 450.27\,\mathrm{Hz}, i.e. the frequency of the vortex shedding (see Figure b). The spatial structures are complex and can be plotted as done for the Fourier modes. The reader finds the scripts in the provided Exercise_6.py file. The spatial structures are nearly indistinguishable from Fourier modes.

In terms of convergence, the DMD does not converge. The temporal structures of this decomposition are not even orthogonal, despite the provided case exhibiting strong periodicity. For a more extensive discussion on the convergence problems of the algorithm, the reader is referred to Mendez et al. [37, 38]; Mendez [34].

DMD spectra in the complex plane

(a)

Absolute values of the DMD eigenvalues as a function of frequency

(b)

Figure 17. Fig (a): DMD spectra for the TR-PIV dataset in tk≤1t_k \le1, plotted in the complex plane together with the unitary circle. Fig (b): absolute values of the DMD eigenvalues as a function of the corresponding frequency in Hertz. The modes with ∣λ∣≈1|\lambda| \approx1 correspond to the vortex shedding.

Beyond Classic Decompositions

The previous sections gave a brief overview of the ‘classic’ decompositions. We here briefly discuss two possible extensions.

The Multiscale Proper Orthogonal Decomposition

In the POD, the temporal structures ψP(tk)\psi_P(t_k) are eigenvectors of the temporal correlation matrix. These have unconstrained frequency content derived under the energy optimality criterium. As a result (see exercise 4), this decomposition finds an effective representation of the data, but the spectra of each mode are broad. In the DMD, the temporal structures ψD(tk)\psi_D(t_k) are eigenvectors of the reduced propagator S~\widetilde{\mathbf{S}} which best approximate the dataset as a linear dynamical system. These have one (complex) frequency, i.e. ψDr(tk)=exp⁡(jωrtk)\psi_{D_r}(t_k)=\exp(j\omega_r t_k) with ωr∈C\omega_r \in\mathbb{C}. As a result (see exercise 5), this decomposition finds the ‘optimal’ frequencies, but the resulting decomposition has convergence problems even in simple datasets.

The limits of these two ‘extreme approaches’ (i.e. ‘energy optimality at all costs’ or ’one-frequency per mode’ at all costs) have motivated many hybrid methods (see Mendez et al. [37]; Mendez [34] for a brief literature review).

The Multiscale Proper Orthogonal Decomposition, first proposed in Mendez et al. [36] aims at finding a compromise between the energy optimality of the POD and the spectral purity of the DMD. The idea is to introduce the notion of optimality within a prescribed range of frequencies. As extensively illustrated in Mendez et al. [37] and Mendez [34], the mPOD is essentially a combination of POD and Multi-resolution Analysis. The MRA can be conveniently performed on the temporal correlation matrix.

The first step of the mPOD is the partitioning of the temporal correlation matrix K\mathbf{K} into the contribution of MM scales. This is done by a set of FIR filters with impulse response h1,h2…hM\mathbf{h}_1,\mathbf{h}_2 \ldots\mathbf{h}_M:

K=∑m=1MK⊗hm=K(1)+K(2)+…K(M),(41)\mathbf{K}=\sum_{m=1}^{M}\mathbf{K}\otimes\mathbf{h}_m=\mathbf{K}^{(1)}+\mathbf{K}^{(2)}+\ldots\mathbf{K}^{(M)}, \tag*{(41)}

where ⊗\otimes denotes convolution and K(m)\mathbf{K}^{(m)} is correlation matrix for the scale mm. The mPOD temporal structures are eigenfunctions of the different contributions, i.e. K(m)ψr(m)=λr(m)ψr(m)\mathbf{K}^{(m)}\psi_r^{(m)}=\lambda_r^{(m)}\psi_r^{(m)}. The filters in Eq. (41) are designed with two criteria: they have separable impulse response and have non-overlapping band-pass regions in the frequency domain. Then, given the filter’s transfer function

Hm(fi,fj)=1nt2∑l∑nhm(tl,tn)e−2π(fitl+fntk),(42)\mathbf{H}_m(f_i,f_j)=\frac{1}{n_t^2}\sum_l\sum_n h_m(t_l,t_n)e^{-2\pi(f_i t_l+f_n t_k)}, \tag*{(42)}

the product HmHn\mathbf{H}_m\mathbf{H}_n is identically zero for all m≠nm\ne n.

The mPOD use Eq. (41) to approximate the correlation matrix using only filters with unitary transfer function along the diagonal, i.e. Hm(fn,fn)≠0\mathbf{H}_m(f_n,f_n)\ne0. Then, it is possible to show that the eigenvectors of all the correlation matrices K(m)\mathbf{K}^{(m)} are mutually orthogonal. Moreover, their frequency content is limited by the transfer function Hm\mathbf{H}_m of the filter identifying the corresponding scale: this means that structures from different scales have no common frequencies. Given ψ^r(m)(fn)=F{ψr(m)}\hat{\psi}_r^{(m)}(f_n)=\mathcal{F}\{\psi_r^{(m)}\} the Fourier transform of the r-th eigenvectors of K(m)\mathbf{K}^{(m)}, the product ψr(m)ψj(n)\psi_r^{(m)}\psi_j^{(n)} is identically zero for all m≠nm \ne n and ψ^r(m)(fn)=0\hat{\psi}_r^{(m)}(f_n)=0 if Hm(fn,fn)=0\mathbf{H}_m(f_n,f_n)=0.

The temporal basis ψ=[ψ(1),ψ(2)…ψ(M)]\psi=[\psi^{(1)},\psi^{(2)}\ldots\psi^{(M)}] collecting the eigenvectors of all scales is the mPOD basis. This is orthogonal by construction and spans Rnt\mathbb{R}^{n_t}; it thus allows for a lossless decomposition while having modes that are optimal within the prescribed range of frequencies.

The associated spatial structures are finally computed via projection and normalization, as done for the DFT in (31) or the POD13 in (34):

σMrΦ⃗Mr(xi)=⟨u⃗~(xi,t),ψr(t)⟩T=∑k=1ntu⃗~(xi,tk)ψMr(tk)(43)\sigma_{Mr}\vec{\Phi}_{Mr}(x_i)=\langle\tilde{\vec{u}}(x_i,t),\psi_r(t)\rangle_T=\sum_{k=1}^{n_t}\tilde{\vec{u}}(x_i,t_k)\psi_{Mr}(t_k) \tag*{(43)}

with σMr=∥⟨u⃗~(xi,t),ψr(t)⟩T∥2\sigma_{Mr}=\left\|\langle\tilde{\vec{u}}(x_i,t),\psi_r(t)\rangle_T\right\|_2.

The software package MODULO provides an efficient implementation of the mPOD given a frequency splitting vector. We use it in the next exercise.

Exercise 7: The mPOD Modes of the TR-PIV data

Compute the mPOD of the full dataset (i.e. nt=13200n_t=13200, corresponding to tkt_k). Consider the following frequency band (in Hz): F_V=np.array([10,290,320,430,470]).

In practice, we look for mPOD modes that have frequency content limited by the entries in F_V. Modes that have frequency content in the range 10-290 Hz should not have frequency content in the range 290-320 Hz, etc. This decomposition has six scales: 0-10, 10-290 … until 470-1500 Hz.

The goal is to identify the structures associated with the first stationary phase, hence a vortex shedding at fn=450f_n=450 Hz, from the ones associated with the second stationary phase, shedding at fn=303f_n=303 Hz. Moreover, we are interested in the large scale evolution of the flow, which we consider describable with modes having frequency fn≤10f_n\leq10 Hz. This explains the choice of F_V.

In this exercise, the reader computes the spatial structures, the temporal structures and the Fourier spectra of the modes linked to the two shedding frequencies.

Solution. Assuming that the data is arranged in a snapshot matrix like in the previous exercises, the mPOD with the prescribed parameters is computed as follows

1 m = MODULO(data=D, MEMORY_SAVING=True, n_Modes = n_r,
2     N_PARTITIONS=11)
3 # Prepare (partition) the dataset
4 D = m._data_processing() # Note that this will cancel D!
5 # mPOD Settings
6 # Frequency splitting Vector
7 F_V=np.array([10,290,320,430,470])
8 # This vector collects the length of the filter kernels.
9 Nf=np.array([1201,801,801,801]);
10 # Decide which scales to keep
11 Keep=np.array([1,1,0,1,0])
12 # Others
13 Ex=1203; boundaries = ’nearest’; MODE = ’reduced’
14 Phis, Psis, Sigmas = m.compute_mPOD(Nf, Ex, F_V, Keep,

15 boundaries, MODE, 1/Fs)

The first four lines are the same as for all decompositions in MODULO. Line 7 prescribes the frequency splitting vector proposed in the exercise. MODULO operates with FIR filter [19] and the vector NfN_f collects the length of the associated impulse responses (which are computed by default using Hamming windows). Line 11 defines the ‘Keep’ vector: if 1, the corresponding scale is kept; if 0, it is skipped. By setting some entries of this vector to 0, the mPOD also acts as a filter in the frequency domain.

The last options, in line 13, prescribe the maximum extension of the signals (ExEx) during the filtering and the necessary boundary extension for the filtering (the convolutions in (41)): in this case, the option ‘nearest’ pads the signals with the last or the first entry. Finally, the mode ’reduced’ controls whether the basis to export is a reduced one or a complete one. The results are stored as a matrix factorization like the POD, and hence the rest of the exercise is a plotting exercise no different from Exercise 5. We thus skip the coding and show the results in Figure 18.

Six-panel visualization of the mPOD modes in spatial, temporal, and frequency domains

Figure 18.

Image not extracted

Figure 18 (PDF p. 40). Figure (a) and (b). Spatial structures of the two modes linked to the vortex shedding during the two stationary conditions. The temporal structures in (c) and (d) show that the modes are active within certain time intervals, corresponding to the stationary phases. The Fourier spectra of the temporal structures (e) and (f) show that these modes are also localized in the frequency domain.

As expected, the mPOD modes capture the modes for the shedding frequencies in the different time intervals. Interestingly, the mPOD inherits from its MRA architecture the ability to localize modes in the frequency domain (these have a relatively narrow frequency bandwidth) with the ability to localize modes in the time domain (these are active only within a specific time interval).

The Kernel PCA

In exercise 5, the reader experienced the remarkable compression capabilities of the POD. With only three modes, it is possible to build an approximation of a complex flow with an l2l_2 error of σP4/σP1=0.1797\sigma_{P4}/\sigma_{P1}=0.1797. This is the best possible result for any linear decomposition.

What if we move to a nonlinear framework? Nonlinear dimensionality reduction is a well-established subfield of machine learning (Alpaydin, [3]; Bishop, [9]), but its application to fluid mechanics is at its infancy (see Noack et al. [42]; Ahmed et al. [2]; Mendez et al. [39]). As for all the other sections, the field is too broad to be explored in this lecture, but a glance at the general ideas might be given on one page. Moreover, powerful libraries such as scikit-learn (Pedregosa et al., [48]) make many nonlinear decompositions accessible in just one line of code.

We consider the most popular nonlinear tool for dimensionality reduction: the Kernel Principal Component Analysis (kPCA) by Schölkopf et al. [58]. Instead of working with the snapshot matrix D\mathbf{D}, we work with a kernelized version of it: ξ(D)∈RnF×nt\xi(\mathbf{D})\in\mathbb{R}^{n_F\times n_t}.

Here ξ(⋅)\xi(\cdot) is a kernel function which maps the snapshots u~(xi,tk)∈Rns\widetilde{\mathbf{u}}(x_i,t_k)\in\mathbb{R}^{n_s} into the so-called feature space ξ(u~)(xi,tk)∈RnF\xi(\widetilde{\mathbf{u}})(x_i,t_k)\in\mathbb{R}^{n_F}. While the spatial structures of the POD (PCA in machine learning terminology) ϕP\boldsymbol{\phi}_P are eigenvectors of DDT\mathbf{D}\mathbf{D}^{T}, the spatial structures of the kPCA (or kPOD, in a sense) ϕK\boldsymbol{\phi}_K are eigenvectors of ξ(D)ξ(D)T\xi(\mathbf{D})\xi(\mathbf{D})^{T}. These are (orthonormal) nonlinear functions of the dataset and their evolution in time (the temporal structures ψK\psi_K) can be computed by projecting the data onto this basis.

If we focus on the forward problem, i.e. the compression problem, we are looking for the projection of the datasets onto the spatial structures ϕK\boldsymbol{\phi}_K. If the kernel functions are well-chosen, these projections can be computed without knowing the ϕK\boldsymbol{\phi}_K's (Bishop, [9])!

The trick to perform this computation is known as kernel trick (Bishop, [9]; Schölkopf et al., [58]) and goes as follows. Let us write the ϕKr\boldsymbol{\phi}_{Kr} as linear combinations of features:

ϕKr=∑i=1npar(tk)ξ(u~)(xi,tk)=ξ(D)ar.(44)\boldsymbol{\phi}_{Kr}=\sum_{i=1}^{n_p}a_r(t_k)\xi(\widetilde{\mathbf{u}})(x_i,t_k)=\xi(\mathbf{D})\mathbf{a}_r. \tag*{(44)}

where ar∈Rnt\mathbf{a}_r\in\mathbb{R}^{n_t} is the vector collecting the coefficients of the combination. Introduce this ansatz into the eigenvalue problem and multiply by ξ(D)T\xi(\mathbf{D})^{T} on both sides:

ξ(D)T(ξ(D)ξ(D)T)ξ(D)ar=λrξ(D)Tξ(D)ar→K2ar=λrKar.(45)\xi(\mathbf{D})^{T}\left(\xi(\mathbf{D})\xi(\mathbf{D})^{T}\right)\xi(\mathbf{D})\mathbf{a}_{r}=\lambda_{r}\xi(\mathbf{D})^{T}\xi(\mathbf{D})\mathbf{a}_{r}\to\mathbf{K}^{2}\mathbf{a}_{r}=\lambda_{r}\mathbf{K}\mathbf{a}_{r}. \tag*{(45)}

where K=ξ(D)Tξ(D)\mathbf{K}=\xi(\mathbf{D})^{T}\xi(\mathbf{D}) collects the inner products in the feature space. If the kernel function is carefully chosen Bishop [9], the inner products defining these matrix can be computed via a kernel function:

Ki,j=ξ(di)Tξ(dj)=κ(di,dj),(46)K_{i,j}=\xi(\mathbf{d}_{i})^{T}\xi(\mathbf{d}_{j})=\kappa(\mathbf{d}_{i},\mathbf{d}_{j}), \tag*{(46)}

where di\mathbf{d}_{i} and dj\mathbf{d}_{j} are two columns of D\mathbf{D}, i.e. two snapshots of the data. If this matrix is invertible, equation (45) reduces to Kar=λrar\mathbf{K}\mathbf{a}_{r}=\lambda_{r}\mathbf{a}_{r}, i.e. the eigendecomposition of K\mathbf{K}. Given the eigenvectors ar\mathbf{a}_{r}, the projection of a feature vector ξ(xi)\xi(\mathbf{x}_{i}) onto the principal component ϕKr\phi_{Kr} in the feature space is:

σKrψKrT=ϕKrTξ(D)=arTξ(D)Tξ(D)=arTκ(D,D).(47)\sigma_{Kr}\psi_{Kr}^{T}=\phi_{Kr}^{T}\xi(\mathbf{D})=\mathbf{a}_{r}^{T}\xi(\mathbf{D})^{T}\xi(\mathbf{D})=\mathbf{a}_{r}^{T}\kappa(\mathbf{D},\mathbf{D}). \tag*{(47)}

Computing these temporal structures is the forward kPCA problem. Reconstructing the dataset from these (i.e. the inverse kPCA), on the other hand, require the computation of the ϕK\phi_{K} and this is a complex nonlinear optimization problem. The reader is referred to Bakir et al. [6] for more information. In the next exercise, we close this section by showing how to compute the kPCA of the provided TR-PIV dataset.

Exercise 8: The kPCA Manifold of the TR-PIV data

Compute the best 3 dimensional manifold approximating the full dataset using forward kPCA. Then, use this manifold to retrieve back an approximation of the velocity field. What is the least square error of the approximation?

Solution. As usual, assuming that the data is arranged in a snapshot matrix as for the previous exercises, this exercise can be solved in one line of code. Scikit-learn provides a powerful function for kPCA:

1  D_s=D.T
2  from sklearn.decomposition import KernelPCA
3  kpca = KernelPCA(kernel="rbf", n_components=3,
4  fit_inverse_transform=True,
5  alpha=1,gamma=None)
6  Psi_K = kpca.fit_transform(D_s)

The convention used by scikitlearn is that of having data realizations (in our case snapshots of the TR-PIV measurement) along the rows of the feature matrix; hence the transposition. The kPCA is here performed using Gaussian Radial Basis functions (see Bishop [9]; Mendez et al. [39]) as kernel functions, with shape factor γ\gamma taken as a default (i.e. 1/ns1/n_s in this case). The call to the function is performed in two steps: line 3 create the kpca object while line 5 runs the function ‘fit_transform’ within the object. The parameter α\alpha is a regularization parameter used to compute the inversion (as described in Bakir et al. [6]). The plot of the 3D manifold derived by the kPCA is simply:

1  fig = plt.figure(figsize=(7,4))
2  ax = fig.add_subplot(111, projection='3d')
3  ax = Axes3D(fig, elev=-150, azim=48)
4  ax.dist = 13
5 ax.scatter(Psi_K[0:n_t:2,1],
6 -Psi_K[0:n_t:2,2],
7 Psi_K[0:n_t:2,0], 'ko')

This is shown in Figure 5.2. The plot of these structures is made to have the same orientation as the one obtained via POD.

Evolution of the reduced order dynamical system in the kPOD basis

Figure 19. Evolution of the reduced order dynamical system in the kPOD basis.

It is thus possible to appreciate that the manifold is axisymmetric and that the system’s orbits remain circular in both the stationary phases. However, the nonlinear mapping results in a different shape, with much narrower orbits at the highest velocity and much larger at the highest (see animations in the presentation). The transitory is also different. The most remarkable result is yet to come: the l2l_2 error obtained by mapping this manifold back to R3\mathbb{R}^3 can be computed as follows:

1 X_rec_kpca = kpca.inverse_transform(Psi_K)
2 Error_kpca=np.linalg.norm(D_s-X_rec_kpca)/np.linalg.norm(D_s)
3 print(Error_kpca)

The reader should find the result in the console: 2.983e-5. This is practically zero and six orders of magnitudes smaller than the error produced by a POD approximation of the same dimension. Somehow, the manifold in Figure 5.2 contains the same amount of information available in the 4260 dimensional space where the TR-PIV velocity fields live and evolve. Isn’t this remarkable?

Final Remarks

These notes offered a tour de force into signal processing, modal analysis and dimensionality reduction to process spatiotemporally resolved data like the one produced by modern image velocimetry. The lecture opened with statistical tools for time series analysis (§ 4), moved to classic modal analysis (§ 5) and explored modern extensions such as multiscale data-driven decompositions and nonlinear (kernel-based) methods (§ 6).

While the topics in Section 3 are textbook material for students in fluid dynamics for at least 30 years, the topics in Section 4 have become standard tools in the researcher’s toolbox more recently. It is thus not surprising that all statistical tools presented in Section 3 have already been integrated into the theory and the modelling of fluid flows, while the same is not entirely true, yet, for the topics in Section 4. However, the trends are clear: today, these are popular tools to extract patterns, to construct simplified representations, or to filter experimental (and numerical) data. The reader is encouraged to dive into the provided exercises to gain computational proficiency on the subject. Finally, the topics in Section 4 are at the forefront of fluid dynamics research and the ever-growing intersection between signal processing and machine learning. The field is very active, and the provided notes and exercises give a good starting point.

References

  1. [1]Adrian, R. J. (2005). Twenty years of particle image velocimetry. Experiments in Fluids, 39(2):159–169.DOI
  2. [2]Ahmed, S. E., Pawar, S., San, O., Rasheed, A., Iliescu, T., and Noack, B. R. (2021). On closures for reduced order models—a spectrum of first-principle to machine-learned avenues. Physics of Fluids, 33(9):091301.DOI
  3. [3]Alpaydin, E. (2014). Introduction to Machine Learning. The MIT Press.
  4. [4]Ayegba, P. O. and Edomwonyi-Otu, L. C. (2020). Turbulence statistics and flow structure in fluid flow using particle image velocimetry technique: A review. Engineering Reports, 2(3).DOI
  5. [5]Baj, P., Bruce, P. J. K., and Buxton, O. R. H. (2015). The triple decomposition of a fluctuating velocity field in a multiscale flow. Physics of Fluids, 27(7):075104.
  6. [6]Bakir, G., Weston, J., and Bernhard, S. (1999). Learning to find pre-images. In Proceedings of the 16th International Conference on Neural Information Processing Systems.
  7. [7]Benner, P., Gugercin, S., and Willcox, K. (2015). A survey of projection-based model reduction methods for parametric dynamical systems. SIAM Review, 57(4):483–531.DOI
  8. [8]Berkooz, G., Holmes, P., and Lumley, J. L. (1993). The proper orthogonal decomposition in the analysis of turbulent flows. Annual Review of Fluid Mechanics, 25(1):539–575.
  9. [9]Bishop, C. M. (2006). Pattern Recognition and Machine Learning. Springer-Verlag New York Inc.
  10. [10]Borodin, A. N. and Salminen, P. (2002). 7. ornstein–uhlenbeck process. pages 528–570. Birkhäuser Basel.DOI
  11. [11]Bruun, H. H. (1995). Hot-Wire Anemometry: Principles and Signal Analysis. OXFORD UNIV PR.
  12. [12]Chen, K. K., Tu, J. H., and Rowley, C. W. (2012). Variants of dynamic mode decomposition: Boundary condition, koopman, and fourier analyses. Journal of Nonlinear Science, 22(6):887–915.DOI
  13. [13]Davidson, P. A. (2004). Turbulence : an introduction for scientists and engineers. Oxford University Press, Oxford, UK New York.
  14. [14]Dawson, S. (2020). Data Driven Fluid Mechanics: Combining First Principles and Machine Learning, chapter The Proper Orthogonal Decomposition. Cambridge University Press; Ed. Miguel Alfonso Mendez and Andrea Ianiro and Bernd Noack and Steve Brunton. Also available as online lecture from the VKI Lecture series ’Machine Learning for Fluid Mechanics, 2020’: https://www.youtube.com/watch?v=IcqBbtWIcIc.
  15. [15]Discetti, S. (2020). Data Driven Fluid Mechanics: Combining First Principles and Machine Learning, chapter Mathematical Tools, Part II: Time-Frequency Analysis. Cambridge University Press; Ed. Miguel Alfonso Mendez and Andrea Ianiro and Bernd Noack and Steve Brunton. Also available as online lecture from the VKI Lecture series ’Machine Learning for Fluid Mechanics, 2020’: https://www.youtube.com/watch?v=54d04zaxmQU.
  16. [16]Emory, M. and Iaccarino, G. (2014). Visualizing turbulence anisotropy in the spatial domain with componentality contours by.
  17. [17]Erik, M. K., Dalibor, C., and M., P. J. (2007). Pod as tool for comparison of piv and les data. In 7th International Symposium on Particle Image Velocimetry, Rome, Italy.
  18. [18]Hasselmann, K. (1988). PIPs and POPs: The reduction of complex dynamical systems using principal interaction and oscillation patterns. Journal of Geophysical Research, 93(D9):11015.DOI
  19. [19]Hayes, M. (2011). Schaums Online of Digital Signal Processing. McGraw-Hill Education - Europe.
  20. [20]Holmes, P., Lumley, J. L., and Berkooz, G. (1996). Turbulence, Coherent Structures, Dynamical Systems and Symmetry. Cambridge University Press.
  21. [21]Holmes, P. J., Lumley, J. L., Berkooz, G., Mattingly, J. C., and Wittenberg, R. W. (1997). Low-dimensional models of coherent structures in turbulence. Phys. Rep., 287(4):337–384.DOI
  22. [22]Hsu, H. (2013). Schaum’s Outline of Signals and Systems, 3rd Edition (Schaum’s Out-lines). McGraw-Hill Education.
  23. [23]Ianiro, A. (2020). Data Driven Fluid Mechanics: Combining First Principles and Machine Learning, chapter Good Practice and Applications of Data-Driven Modal Analysis. Cambridge University Press; Ed. Miguel Alfonso Mendez and Andrea Ianiro and Bernd Noack and Steve Brunton. Also available as online lecture from the VKI Lecture series ’Machine Learning for Fluid Mechanics, 2020’: https://www.youtube.com/watch?v=H6twKFTCd2k.
  24. [24]Johansson, R. (2018). Numerical Python. Apress.
  25. [26]Kay, S. M. (1993). Fundamentals of Statistical Processing, Volume I. Prentice Hall.
  26. [27]Langtangen, H. P. (2016). A Primer on Scientific Programming with Python. Springer-Verlag GmbH.
  27. [28]Lavoie, P., Avallone, G., Gregorio, F. D., Romano, G. P., and Antonia, R. A. (2007). Spatial resolution of PIV for the measurement of turbulence. Experiments in Fluids, 43(1):39–51.DOI
  28. [29]Ljung, L. and Glad, T. (1994). Modeling of Dynamic Systems. Prentice Hall.
  29. [30]Lumley, J. L. (1970). Stochastic Tools in Turbulence. DOVER PUBN INC.
  30. [31]Lumley, J. L. and Newman, G. R. (1977). The return to isotropy of homogeneous turbulence. Journal of Fluid Mechanics, 82(1):161–178.
  31. [32]Mcdonough, J. M. (2004). Introductory lectures on turbulence physics, mathematics and modeling.
  32. [33]Mendez, M., Raiola, M., Masullo, A., Discetti, S., Ianiro, A., Theunissen, R., and Buchlin, J.-M. (2017). POD-based background removal for particle image velocimetry. Experimental Thermal and Fluid Science, 80:181–192.DOI
  33. [34]Mendez, M. A. (2020a). Data Driven Fluid Mechanics: Combining First Principles and Machine Learning, chapter Generalized and Multiscale Data-Driven Modal Analysis. Cambridge University Press; Ed. Miguel Alfonso Mendez and Andrea Ianiro and Bernd Noack and Steve Brunton. Also available as online lecture from the VKI Lecture series ’Machine Learning for Fluid Mechanics, 2020’: https://www.youtube.com/watch?v=BJQzk03HaTk.
  34. [35]Mendez, M. A. (2020b). Data Driven Fluid Mechanics: Combining First Principles and Machine Learning, chapter Mathematical Tools, Part I: Continuous and Discrete LTI Systems. Cambridge University Press; Ed. Miguel Alfonso Mendez and Andrea Ianiro and Bernd Noack and Steve Brunton. Also available as online lecture from the VKI Lecture series ’Machine Learning for Fluid Mechanics, 2020’: https://www.youtube.com/watch?v=qvZmKr6fhW4.
  35. [36]Mendez, M. A., Balabane, M., and Buchlin, J.-M. (2018a). Multi-scale proper orthogonal decomposition (mPOD). In AIP Conference Proceedings.DOI
  36. [37]Mendez, M. A., Balabane, M., and Buchlin, J.-M. (2019a). Multi-scale proper orthogonal decomposition of complex fluid flows. Journal of Fluid Mechanics, 870:988–1036.DOI
  37. [38]Mendez, M. A., Hess, D., Watz, B. B., and Buchlin, J.-M. (2020). Multiscale proper orthogonal decomposition (mPOD) of TR-PIV data—a case study on stationary and transient cylinder wake flows. Measurement Science and Technology, 31(9):094014.DOI
  38. [39]Mendez, M. A., Pino, F., and Fiore, M. (2019b). Optimization Methods for Computational Fluid Dynamics, chapter Machine Learning for Fluid Mechanics: Challenges, Opportunities and Perspectives. von Karman Institute for Fluid Dynamics LS.
  39. [41]Ninni, D. and Mendez, M. A. (2020). MODULO: A software for multiscale proper orthogonal decomposition of data. Software X, 12:100622.arxiv.org/abs/2004.12123
  40. [42]Noack, B., Ehlert, A., Nayeri, C., and Morzynski, M. (2020). Data Driven Fluid Mechanics: Combining First Principles and Machine Learning, chapter Mathematical Tools, Part II: Time-Frequency Analysis. Cambridge University Press; Ed. Miguel Alfonso Mendez and Andrea Ianiro and Bernd Noack and Steve Brunton. Also available as online lecture from the VKI Lecture series ’Machine Learning for Fluid Mechanics, 2020’: https://www.youtube.com/watch?v=iehMMhDqm ys.
  41. [43]Øksendal, B. (1998). Stochastic Differential Equations. Springer Berlin Heidelberg.
  42. [44]Oliveira, T. F., Misserda, R. B., and Cunha, F. R. (2007). Dynamical simulation and statistical analysis of velocity fluctuations of a turbulent flow behind a cube. Mathematical Problems in Engineering, 2007:1–28.DOI
  43. [45]Oppenheim, A. V. (2015). Signals, Systems and Inference. Pearson Education.
  44. [46]Oppenheim, A. V., Willsky, A. S., and with S. Hamid (1996). Signals and Systems (2nd Edition). Pearson.
  45. [47]Paul J. Deitel, H. D. (2019). Intro to Python for Computer Science and Data Science: Learning to Program with Ai, Big Data and the Cloud. PEARSON.
  46. [48]Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cour­napeau, D., Brucher, M., Perrot, M., and Duchesnay, E. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830.arxiv.org/abs/1201.0490
  47. [49]Penland, C. (1996). A stochastic model of IndoPacific sea surface temperature anomalies. Physica D: Nonlinear Phenomena, 98(2-4):534–558.DOI
  48. [50]Penland, C. and Magorian, T. (1993). Prediction of niño 3 sea surface temperatures using linear inverse modeling. Journal of Climate, 6(6):1067–1076.doi.org/10.1175/1520-0442(1993)006<1067:ponsst>2.0.co;2
  49. [51]Pope, S. B. (2000). Turbulent Flows. Cambridge University Press.
  50. [52]Raffel, M., Willert, C. E., Scarano, F., Kähler, C. J., Wereley, S. T., and Kompenhans, J. (2018). Particle Image Velocimetry. Springer International Publishing.
  51. [53]Raiola, M., Discetti, S., and Ianiro, A. (2015). On PIV random error minimization with optimal POD-based low-order reconstruction. Experiments in Fluids, 56(4).DOI
  52. [54]Rowley, C., Mezić, I., Bagheri, S., Schlatter, P., and Henningson, D. (2009). Spectral analysis of nonlinear flows. J. Fluid Mech., 641:115.
  53. [55]Saarenrinne, P. and Piirto, M. (2000). Turbulent kinetic energy dissipation rate estimation from PIV velocity vector fields. Experiments in Fluids, 29(7):S300–S307.DOI
  54. [56]Scharnowski, S., Bross, M., and Kähler, C. J. (2018). Accurate turbulence level estimations using PIV/PTV. Experiments in Fluids, 60(1).DOI
  55. [57]Schena, L. and Mendez, M. A. (2020). A python package for modulo (modal multiscale pod). https://modulo.readthedocs.io/en/latest/intro.html. Accessed: 2021-11-10.
  56. [58]Schölkopf, B., Smola, A., and Müller, K.-R. (1997). Kernel principal component analysis. In Lecture Notes in Computer Science, pages 583–588. Springer Berlin Heidelberg.
  57. [59]Schmid, P. (2010). Dynamic mode decomposition of numerical and experimental data. J. Fluid Mech., 656:5–28.
  58. [60]Segalini, A., Bellani, G., Sardina, G., Brandt, L., and Variano, E. A. (2014). Corrections for one- and two-point statistics measured with coarse-resolution particle image velocimetry. Experiments in Fluids, 55(6).DOI
  59. [61]Simonsen, A. J. and Krogstad, P.-Å. (2005). Turbulent stress invariant analysis: Clarification of existing terminology. Physics of Fluids, 17(8):088103.DOI
  60. [62]Sirovich, L. (1987). Turbulence and the dynamics of coherent structures: Part i. coherent structures. Quart. Appl. Math., 45(3):561–571.DOI
  61. [63]Smith, J. O. (2007a). Introduction to Digital Filters: with Audio Applications. W3K Publishing.
  62. [64]Smith, J. O. (2007b). Mathematics of the Discrete Fourier Transform (DFT): with Audio Applications. W3K Publishing.
  63. [66]Svein Linge, H. P. L. (2019). Programming for Computations - Python. Springer International Publishing.
  64. [67]Tennekes, H. (1972). A first course in turbulence. The MIT Press, Cambridge, Massachusetts.
  65. [69]Uruba, V. (2012). Decomposition methods in turbulence research. EPJ Web of Conferences, 25:01095.DOI
  66. [70]von Storch, H. and Xu, J. (1990). Principal oscillation pattern analysis of the 30- to 60-day oscillation in the tropical troposphere. Climate Dynamics, 4(3):175–190.DOI
  67. [71]Wang, G., Yang, F., Wu, K., Ma, Y., Peng, C., Liu, T., and Wang, L.-P. (2021). Estimation of the dissipation rate of turbulent kinetic energy: A review. Chemical Engineering Science, 229:116133.DOI
  68. [72]Wang, R. (2009). Introduction to Orthogonal Transforms. Cambridge University Press.
  69. [73]Welch, P. (1967). The use of fast fourier transform for the estimation of power spectra: A method based on time averaging over short, modified periodograms. IEEE Transactions on Audio and Electroacoustics, 15(2):70–73.
  70. [74]Williamson, D. (1999). Discrete-time Signal Processing. Springer London.

Paper details

vlnc:W_vs7e4jt3
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

Not recorded for this paper

View statement
Looking for related papers…
Contents