WorksPhysics and Astronomy
Statistical Treatment, Fourier and Modal Decomposition
Abstract
These are lecture notes for the lecture "Statistical Treatment, Fourier and Modal Decompositions", given at the VKI Lecture series "Fundamentals and Recent Advances in Particle Image Velocimetry and Lagrangian Particle Tracking". The course was held at the von Karman Institute for fluid dynamics from 15 November to 18 November 2021. This lecture provides a guided tour through the processing of data acquired via image velocimetry. Far from being an exhaustive account of the field, which would require an entire course on its own, the scope is to provide a hands-on tutorial. This begins with basic statistical treatment, briefly reviews frequency and modal analysis, and conclude with more advanced research topics such as multiscale modal decompositions and nonlinear dimensionality reduction. The material covered should hopefully propel newcomers into the subject while remaining of interest to experienced practitioners. All the codes related to this lecture are made available on a github repository.
How to cite this work
1 @InProceedings{Mendez2020,
2 author = {Miguel A. Mendez},
3 title = {Statistical Treatment, Fourier and Modal Decompositions},
4 booktitle = {Fundamentals and Recent Advances in Particle
5 Image Velocimetry and Lagrangian Particle Tracking, VKI Lecture Series},
6 editor = {Discetti, S. and Mendez, M. A},
7 year = {2021},
8 publisher = {von Karman Institute},
9 ISBN = {978-2-87516-181-9},
10 }
Lecture Format and Organization
Modern image velocimetry produces data with spatio-temporal resolutions that were inconceivable a couple of decades ago [52, 1]. This lecture is about processing that data and reviews a broad range of techniques.
With the intent of being accessible while providing some hands-on experience, the lecture is cast a sequence of exercises in Python. Thus, these notes should be read while keeping a Python terminal at hand, ready to run the provided scripts. Python is nowadays the lingua franca of data processing and “soft computing”; it is an extremely powerful language, it is free, and it is easy to use. While the reader might not be an experienced programmer, it is hoped that the presented scripts are simple enough to be intelligible. Running and interacting with these is essential to the correct delivery of the presented material. Readers interest in learning Python for scientific computing are referred to Svein Linge [66]; Langtangen [27]; Johansson [24]; Paul J. Deitel [47].
All the presented tools will be applied to a specific time-resolved PIV dataset. The details on this dataset and the instructions to download it are provided in § 3. The provided exercises guide the reader to the implementation of the most important data processing operations in statistical analysis, frequency analysis and modal decompositions. The theoretical background is kept to a minimum, and the reader is urged to consult the provided references to learn more.
The first part, in § 3, covers some essential statistical treatment, primarily focusing on time series analysis. There we treat the dataset as a set of independent time series corresponding to the velocity signal in various positions. These time series are, of course, not independent. However, the methods presented in this section do not explore nor exploit their spatial dependency. The tools introduced in this section are also the oldest: these were the only ones covered in data processing courses before image velocimetry made space and time resolution easily accessible. The second part, in § 4, moves from a “local processing” to a “global processing”, i.e. including both space and time analysis. This takes us to modal analysis, and the section reviews the notion of flow modes and modal decompositions. These are introduced first for the familiar Discrete Fourier Transform and then for the two most popular data-driven decompositions: the Proper Orthogonal Decomposition (POD) and the Dynamic Mode Decomposition (DMD). We shall explore “pro” and “cons” of each with the hope that the reader quickly realizes that the distinction between “pro” and “cons” is meaningless: these are simply different tools that serve different purposes and answer different questions.
The last part, in § 6, covers two advanced research topics. The first topic is an extension of the previous methods while keeping their linear framework: a recent decomposition that breaks the dichotomy POD/DMD and proposes a bridge between the two. This is the multi-scale Proper Orthogonal Decomposition (mPOD), extensively described in Mendez et al. [37]; Mendez [34]. The second topic also extends the previous methods but brings the treatment to a nonlinear framework, with kernels functions defining nonlinear similarity metrics. This section touches briefly on nonlinear dimensionality reduction and manifold learning. These are well-known tools in machine learning [9] but are now gaining popularity also in fluid mechanics, offering new ways of looking at the dynamics of fluid flows. § 7 closes with some final remarks.
Collecting the Dataset
The selected test case is the flow past a cylinder of diameter and length in transient conditions, with varying free stream velocity. The experiments were carried out in the L10 low-speed wind tunnel of the von Karman Institute, instrumented with a TR-PIV system from Dantec Dynamics. The details of the experiment are described in Mendez et al. [38]. The dataset can be downloaded by the provided script Get_Cylinder_DATA.py, and particularly the following lines:
import urllib.request
print(’Downloading Data PIV for Chapter 10... ’)
url = ’https://osf.io/47ftd/download’
urllib.request.urlretrieve(url, ’Ex_5_TR_PIV_Cylinder.zip’)
print(’Download Completed! I prepare data Folder’)
# Unzip the file
from zipfile import ZipFile
String=’Ex_5_TR_PIV_Cylinder.zip’
zf = ZipFile(String,’r’);
zf.extractall(’./DATA_CYLINDER’); zf.close()
The dataset consists of velocity fields, sampled at over a grid of points. The spatial resolution is approximately . A plot of an exemplary time step, e.g. time step 10, can be printed as follows:
import os
import matplotlib.pyplot as plt
from Functions_Chapter_10 import Plot_Field_TEXT_Cylinder
FOLDER=’DATA_CYLINDER’
Name=FOLDER+os.sep+’Res%05d’%10+’.dat’ # Check it out: print(Name)
Name_Mesh=FOLDER+os.sep+’MESH.dat’
n_s,Xg,Yg,Vxg,Vyg,X_S,Y_S=Plot_Field_TEXT_Cylinder(Name,Name_Mesh)
Name=’Snapshot_Cylinder.png’
plt.savefig(Name, dpi=200)
The figure produced from this script is shown in Figure 1. Three probes, denoted as P1, P2 and P3, are placed to extract time series data for the following exercises. The location of these probes is also shown in the plot.

Figure 1. Example snapshot for the dataset analyzed in this lecture. This is the flow past a cylinder in transient condition from Mendez et al. [38]. In the following section, the time series obtained at three virtual probes P1, P2, P3 will be analyzed. The location of these probes is shown in the figure.
The following code extracts the time series of the velocity in the three locations:
#%% Get Time Series
import numpy
import numpy as np
# Define the location of the probes in the grid
i1,j1=2,4; xp1=Xg[i1,j1]; yp1=Yg[i1,j1]
i2,j2=14,18; xp2=Xg[i2,j2]; yp2=Yg[i2,j2]
i3,j3=23,11; xp3=Xg[i3,j3]; yp3=Yg[i3,j3]
# Prepare the time axis
n_t=13200; Fs=3000; dt=1/Fs
t=np.linspace(0,dt*(n_t-1),n_t) # prepare the time axis#
# Initialize Probes P1,P2,P3
P1=np.zeros(n_t); P2=np.zeros(n_t); P3=np.zeros(n_t);
# number of velocity points
#from Plot_Field_TEXT_Cylinder)
nxy=int(n_s/2)
for k in range(0,n_t):
# Name of the file to read
Name=FOLDER+os.sep+'Res%05d'%(k+1)+'.dat'
# Read data from a file
# Here we have the two columns
DATA = np.genfromtxt(Name,usecols=np.arange(0,2),max_rows=nxny+1)
Dat=DATA[1:,:] # Remove the first raw with the header
# Get U component and reshape as grid
V_X=Dat[:,0]; V_X_g=np.reshape(V_X,(n_y,n_x))
# Get V component and reshape as grid
V_Y=Dat[:,1]; V_Y_g=np.reshape(V_Y,(n_y,n_x))
# Get velocity magnitude
V_MAGN=np.sqrt(V_X_g**2+V_Y_g**2)
# Sample the three probes
P1[k]=V_MAGN[i1,j1]; P2[k]=V_MAGN[i2,j2]; P3[k]=V_MAGN[i3,j3];
print('Loading Step '+str(k+1)+'/'+str(n_t))
Figure 2 shows the signals in P1 (on the left) and P3 (on the right), together with their histogram and their power spectral densities. The meaning of these quantities will be evident by the end of the lecture. For the moment, it suffices to notice that this test case is characterized by a large scale variation of the free-stream velocity.
Figure 2. Velocity time series in probes P1 (on the left) and P3 (on the right) for the TR-PIV dataset analyzed in this lecture. See Figure 1 for the probe location.
This is evident from the velocity evolution of both probes, but it is clearer in P1 because this probe is sufficiently far from the cylinder wake to capture the evolution of the free-stream flow. In the first 1.5s, the free stream velocity is at approximately . During this time, the flow is stationary (more about this in the next session).
Between and the velocity drops down to , reaching a second stationary phase that last until the end of the acquisition. The variation of the flow velocity is sufficiently low to let the vortex shedding adapt and hence preserve an approximately constant Strouhal number of , with the diameter of the cylinder. Consequently, the vortex shedding varies from to . The two peaks in the PSD in probes P3 clearly shows both frequencies, although a frequency analysis does not allow to identify when these occur. This will also be briefly addressed later. The same peaks are much weaker but still present in P1.
In the histogram of both probes, two distributions are visible. These corresponds to the first () and the second () stationary conditions. The next section is dedicated to statistical analysis time series like these.
Statistics
This section presents some basic concepts and tools to study time series. We introduce the notion of random processes and random variables in the subsection 3.1, along with the relevant first and second-order statistics. As a first exercise, we study the ensemble statistics of an Ornstein Uhlenbeck process before recalling the notion of stationarity in the strong and weak sense.
The subsection 3.2 introduce the notion of ergodicity and the time statistics for finite duration signals. These are linked, in 3.3, to fundamental statistical quantities characterizing turbulent flows, further discussed in the second exercise.
Finally, 3.4 completes the introduction by reviewing the spectral characterization of stochastic signals. The review thus includes the notion of power spectral density and coherency explored in the third exercise.
The material covered falls within well-known textbooks in statistical inference and signal processing. Excellent references with a focus on deterministic signals are Oppenheim et al. [46]; Ljung and Glad [29]; Hsu [22]; Hayes [19], while Smith [64] provides a comprehensive treatment of Fourier analysis and filters. A gentle introduction to stochastic signals is provided by Kay [26]; Williamson [74]; Oppenheim [45] while Wang [72] provides an excellent review of orthogonal transforms.
First and Second Order Statistics
Each of the grid points in the TR-PIV measurements generates a time series of equally spaced samples in time. We will use the notation to refer to the time series generated at the spatial location, with and the discrete time axis. Denoting as the Cartesian grid from which the PIV data is available, with and , we can see as a matrix linear index scanning all the points of the grid. Accordingly, we use to denote the location of the probe.
Unless otherwise stated, the time series under analysis is the velocity magnitude at that location :
In a statistical treatment of signals, we admit that it is not possible (or simply not convenient) to describe signals with a deterministic model: we will not be able (nor we want to attempt) to predict the exact value observed at a certain time. Instead, we will be content with the ability to attribute a probability to .
More concretely, we treat a time series as the realization of a random process, and we treat the value that the time series takes at a specific time as a random variable. A random process can be seen as a family of jointly distributed random variables indexed by the time . Let denote the set of random variables associated to the value of . A full probabilistic characterization requires the joint probability density function of random variables:
This is a -order description, and it is clearly impractical in most cases of interest. It is also not particularly interesting because we are usually not interested in computing the probability that a specific realization of samples occurs.
On the other hand, we are always interested in a first order description. In other words, we seek to predict the probability of observing a certain at a certain time . This is a ’one-point statistics’, governed by the univariate probability density function (pdf) associated to the random variable . This pdf allows to assign the probability that :
Given this distribution, the mean (expectation) of the time series at is:
having introduced the expectation operator and the short hand notation with brackets . Similarly, the variance of the time series at time is:
where is the standard deviation, also referred to as the root mean square of .
In a second order description of the time series, we might be interested in the statistical links between the value that the time series takes at time and the values it takes at time . The expectation of the product is the all important autocorrelation function, whose definition requires the (bivariate) joint probability function :
Similarly, one can define the autocovariance function as
It is easy to note that .
Finally, we might be interested in the statistical link between two different time series; for examples the ones generated at two different locations. Denoting these as and we recall the celebrated cross-correlation (the engine of PIV analysis!), which requires the (bivariate) joint probability function where is the random variable associated to and is the one associated to . We have:
and the cross-covariance is simply
All the above quantities are difficult to evaluate because none of the invoked distributions is usually known. To compute these important statistics, one must then also estimate the underlying distributions, and the only way to do this is to sample many instances of the time series. The set of instances we will be working with is called ensamble and the expectation operator should now be generalized to the notion of ensamble averaging. Let us review this concept in some detail for the mean in (3); we leave the extensions of the others statistics as an exercise. Let us assume that a total of realizations has been collected (each being of length and indexed by ) and let be the realization at time .
Adopting a frequentist approach to probabilities, we can compute the probability that the value is taken at as , where is the number of times such value has been observed at time within the full ensemble3.
Then, replacing the integral with a summation leads to
For reasons that will soon become obvious, it is convenient to construct a snapshot matrix with every column collecting a realization of length of the stochastic process in the probe located at , and every row containing the realizations at time . Every realization, at the moment, must be thought of as a different experiment. With this arrangement of the data, the ensemble average of the series is the average across the rows of . This is equivalent to multiplying the matrix by a vector of length containing all terms. Similarly, the ensemble variance at all times can be computed along all rows of . The computation of both quantities is shown in the first exercise.
Moving to second order statistics, the computation of the autocorrelation function requires averaging all products . The ensemble autocorrelation in the matrix formalism becomes4
where is the row vector collecting all the realizations at time , the superscript denotes transposition. This is essentially an inner product. The autocovariance is simply , and the reader should notice that .
The cross correlation between two variables is analogous to (10): denoting as and the snapshot matrices collecting the sets of ensembles for the two random processes, the cross correlation reads
Finally, it is worth noticing that in many fields, the definitions of , are normalized so that their value is in the range . The normalized quantities are distinguished with a hat and read:
This is no different than the simple arithmetic average in (9).
As a note of warning, the reader should note that in a vast portion of the literature in experimental fluid mechanics, statistics are computed on the ‘fluctuating’ component of the velocity fields, following a classic Reynolds decomposition (Pope, [51]) (see also § 4.7). Therefore, the notion of autocovariance and autocorrelation are used interchangeably.
It is now worth pausing and practice with an exercise.
Exercise 1: The statistics of an Ornstein-Uhlenbeck Process
Consider the Ornstein-Uhlenbeck process governed by the following stochastic differential equation (see Øksendal [43]; Borodin and Salminen [10]; Pope [51])
where the subscript here refers to a time step, is called rate of reversion and controls how quickly the process reaches its (stationary) long-term behaviour, characterized by mean and random fluctuation with standard deviation . The second term is a stochastic term, and it is here taken as a normal Gaussian with zero mean and unitary standard deviation, here written as . The first term is often referred to as drift, the second as diffusion.
Consider a case with initial condition , , and . Consider samples, with a sampling of s. This corresponds to a physical observation time of s. Plot four realizations to explore the process.
The questions are two. (1) compute the ensemble mean, the ensemble standard deviation and the ensemble autocovariance as a function of time using . (2) Study the convergence of these statistical quantities at two time steps (say at and ) as a function of the number of samples in the ensemble.
Solution. Let us begin by creating a function that produces a sample of the process, taking as input the four process parameters , , and vector of times . Using a simple loop, the following function does the job (see Python exercise 1):
1 import numpy as np
2 # Function definition
3 def U_O_Process(kappa,theta,sigma,t):
4 n_T=len(t)
5 # Initialize the output
6 y=np.zeros(n_T)
7 # Define Drift and Diffusion functions in the process
8 drift= lambda y,t: kappa*(theta-y)
9 diff= lambda y,t: sigma
10 noise=np.random.normal(loc=0,scale=1,size=n_T)*np.sqrt(dt)
11 # Solve Stochastic Difference Equation
12 for i in range(1,n_T):
13 y[i]=y[i-1]+drift(y[i-1],i*dt)*dt+diff(y[i-1],i*dt)*noise[i]
14 return yThe following script creates realizations and store them in a matrix
of size . Then, it plots four randomly chosen samples, namely the numbers . The four realizations are shown in Figure 3.1, with the plot axis being customized (see the provided codes).
import matplotlib.pyplot as plt
# Initial and final time
t_0=0; t_end=10
# Number of Samples
n_t=1001
# Process Parameters
kappa=1.2; theta=3; sigma=0.5
# Create the time scale
t=np.linspace(t_0,t_end,n_t); dt=t[2]-t[1]
# Collect 500 sample and store in U_N
n_r=500; U_N=np.zeros((n_t,n_r))
for l in range(n_r):
U_N[:,l]=U_O_Process(kappa,theta,sigma,t)
# Plot the results
plt.figure(1)
plt.plot(t,U_N[:,1])
plt.plot(t,U_N[:,10])
plt.plot(t,U_N[:,22])
plt.plot(t,U_N[:,55])
The reader should play with the code long enough to realize that this process is characterized by a transient time of the order of 0.3s within which all signals move from zero to a stationary conditions (a precise definition of stationary signals is yet to come). Having arranged the data into ‘snapshot matrices’, the computation of the time average and the temporal standard deviation can be done in one line each:
U_Mean=np.mean(U_N,axis=1) # Ensamble Mean
U_STD=np.std(U_N,axis=1) # Ensamble STD
These are essentially ‘row-wise’ statistics.
Figure 3.1 shows the time average together with the range . It appears that the standard deviation grows gently until it reaches a constant value after about s.
Finally, we analyze the ensamble autocorrelation of this random process. The ensamble cross-correlation between the time steps and for the ensambles and of two random variables (see (11)) can be combuted with the following function:
def Norm_Autocov_0(U_N,W_N,k,j):
n_r,n_t=np.shape(U_N)
# Select all realizations at time t_k for U
U_N_k=np.expand_dims(U_N[k,:],axis=0);
# Select all realizations at time t_kj for W
W_N_k=np.expand_dims(W_N[j,:],axis=0)
# Note (These are row vectors)
# Compute the average products
PR=U_N_k.T.dot(W_N_k)
R_UW=np.mean(PR)/(np.std(U_N_k)*np.std(W_N_k))
return R_UW

Figure 3. Fig (a): Four randomly chosen samples of the Ornstein-Uhlenbeck process; Fig (b): Ensamble average of the process (in blue), along with the and curves.
Having added a normalization. Then, the following script computes the autocorrelation of 100 randomly pairs, separated by a lag of 50 samples:
# Define lag (in number of samples)
lag=50
# Study the autocorrelation between two points at equal lags
N_S=100; R_UW=np.zeros(N_S)
# Select a 100 random points i (larger than 500)
J=np.random.randint(500,800,N_S); K=J+50
for n in range(N_S):
R_UW[n]=Norm_Autocov_0(U_N,U_N,J[n],K[n])
The resulting set of autocorrelations has a mean of 0.545 and a standard deviation of 0.0059: in other words, it does not matter what the exact pair is, as long as they differ by the same lag (in this case, 50). This is because here we are sampling in time intervals from 500 to 800, and here the process has reached its stationary condition. The reader is encouraged to repeat the exercise at an earlier interval.
Finally, we analyze the convergence of the statistics as a function of the number of realizations. The following script computes the mean and the standard deviation at and for a number of realizations that goes from 1 to 1000.
n_R=np.round(np.logspace(0.1,3,num=41))
# Prepare the outputs at k=100
mu_10=np.zeros(len(n_R))
sigma_10=np.zeros(len(n_R))
# Prepare the outputs at k=700
mu_700=np.zeros(len(n_R))
sigma_700=np.zeros(len(n_R))
# Loop over all n_R’s.
for n in range(len(n_R)):
# show progress
print('Computing n='+str(n)+' of '+str(len(n_R)))
n_r=int(n_R[n]) # Define the number of ensambles
U_N=np.zeros((n_t,n_r)) # Initialize the ensamble set
for l in range(n_r): # Fill the Ensamble Matrix
U_N[:,l]=U_O_Process(kappa,theta,sigma,t)
# Compute the mean and the std's
mu_10[n]=np.mean(U_N[10,:]) # Ensamble Mean
sigma_10[n]=np.std(U_N[10,:]) # Ensamble STD
mu_700[n]=np.mean(U_N[700,:]) # Ensamble Mean
sigma_700[n]=np.std(U_N[700,:]) # Ensamble STD
By analyzing the vector in the script, the reader should note that some of the entries with a small number of samples are taken multiple times to show the variability in the prediction. The results are shown in Figure 4.

Figure 4. Fig (a): Four randomly chosen samples of the Ornstein-Uhlenbeck process; Fig (b): Ensamble average of the process (in blue), along with the and curves.
It can be shown that the convergence of both the mean and the standard deviation is , but the convergence of the mean is while the convergence of the standard deviation is (see also [see also 23]): this explains why a larger number of samples is needed to converge the second-order statistics, and why the convergence at is slower. The reader is encouraged to explore the provided scripts further, to observe this result in action at different points.
We close this section with the definition of stationarity. A process is stationary in a strict sense (SSS) if all its statistics are invariant to a shift of the origin. In other words, we could add any to the times in (1) and still have the same distribution. A less restrictive concept is that of a process which is stationary in a weak sense (WSS). In this case, only the first and second order statistics are time invariant. As a result, the ensamble mean is constant, i.e. , and the autocorrelation/autocovariance are only function of the time delay between the two points considered: and .
Ergodicity and Finite Durations
Ensamble averaging requires repeating experiments hundreds of times. This is not feasible. However, because stationary signals’ statistics are time-invariant, it is reasonable to wonder whether a sufficiently long experiment can be statistically equivalent to many short ones.
This brings us to the notion of ergodicity. This is a sophisticated concept (see also [30, 45, 11], but the essential idea can be explained as follows: a process is ergodic if ensemble statistics can be replaced by temporal statistics on any particular realization5. We avoid introducing the statistics in the continuous domain and move straight to the statistics for a discrete sequence like the ones encountered in practice. The time average is defined as
while the temporal autocorrelation is
The expectation is now taken in time (i.e. ). A subscript is used to distinguish from the expectation in the ensemble set, and a tilde is used to distinguish the temporal statistics from the ensemble ones. While the definition of the time average is rather obvious, the one of autocorrelation deserves a deeper discussion.
Both quantities converge at sufficiently large for a stationary signal while none of them is meaningful for non-stationary signals. We come back to the convergence problem with an exercise at the end of this section. For the moment, the main concern in the reader’s mind should be a practical one: even assuming that is large enough (hence the limit can be dropped), eq (15) cannot be computed for finite duration signals unless additional information is provided.
Let us illustrate this with an example: compute the autocorrelation of the time series . Consider a Python-like notation, hence and . Considering , we could use an index notation and write or regardless of the . Because this signal has entries, we have a total of possible lags . The lags in (15) take the form , and the autocorrelation will thus be a vector of length 7 which can also be indexed as with . Figure 5 illustrate the problem of computing : this is the correlation between the signal and its copy shifted by 2 entries to the right .

Figure 5. Sketch of the difference between linear and cyclic autocorrelation. The figure shows how these two definitions compute the autocorrelation entry of the vector . The linear correlation pads with zero on both sides while the cyclic correlation assumes periodicity with period .
While it is clear that the summation will contain , it is unclear how to deal with the entries that do not have a match in the other signal because of the finite size of the signal. On a practical level, the two classic solutions are zero padding and cyclic padding.
The zero-padding leads to the linear correlation: the vectors are padded by zeros in all the entries lacking information. The cyclic padding leads to the cyclic correlation: the vectors are assumed periodic with period . We distinguish these operators with and or a , and the results for this example is:
Ergodicity and Finite Durations
Note that is periodic and for both operators we have . This is why one often plots only6 the last vectors of the autocorrelation, corresponding to the lags from to . The same considerations are valid for the computation of the autocovariance, the cross correlations and the cross covariance, and the reader should be able to derive their formulae.
When the time series are ‘short’, and are very different, and it is essential to clarify which is being used. ‘Short’ here means that relevant lags (giving normalized cross-correlation of ) are present within a time scale that is comparable to the duration of the signal. This is the case of PIV interrogation (see Käufer (2020)), in which time is replaced by space, and the discrete-time indices are the shifts in pixels.
The linear (acyclic) operators are unbiased estimators (Kay, [26]) and provide better statistical convergence to their ensemble counterparts. On the other hand, the cyclic operators are computationally more interesting because the periodic assumption enables a link with the cyclic convolution, which can be computed in the frequency domain using the Fast Fourier Transform (FFT). Briefly, it is easy to note that the cyclic convolution between two vectors and differs from the cyclic correlation by the flipping of the second vector (see Hayes [19]). In the frequency domain, this flipping corresponds to the conjugation of the associated Fourier transform.
To be more specific, let be the Discrete Fourier Transform (DFT) of , i.e.:
where , with and the frequency resolution, with the sampling frequency. The cyclic cross-correlation between two signals , , having DFT , can be computed as
with the overbar denoting complex conjugation. A Python (and normalized) implementation of the cross covariance is thus provided by following function:
def R_UW_C(u,v):
RUU=np.fft.ifft(np.abs(np.fft.fft(u)*np.fft.fft(v))).real
c=(RUU/len(u)-(np.mean(u)*np.mean(v)))/(np.std(u)*np.std(v))
return c[:len(u)//2]
The auto-covariance is thus simply while moving from covariances to correlations is only a matter of removing or keeping the time average.
It is possible to compute the linear operators using a circular extension if an appropriate zero padding is used before the cyclic extension (see Smith [64]). This is what software packages like Scipy and Numpy do when computing the ’FFT-based’ cross-correlation. The linear (and normalized) cross-covariance using the Scipy’s function correlate is implemented in the following function:
from scipy import signal
def R_UW_L(u,v):
# Call to the scipy function correlate:
RUU = signal.correlate(u-np.mean(u), v-np.mean(v),\
mode=’same’,method=’auto’)/len(u)/(np.std(u)*np.std(v))
return RUU[RUU.size//2:]
The entry ‘method’, set by default to ‘auto’, uses the fastest option between direct and FFT-based correlation, depending on the size of the array. The reader will need these functions for the next exercise. But first, we might need to recall some more definitions.
A note on Turbulence
The statistical treatment of turbulence is a vast subject, far beyond the scope of this lecture. The reader is referred to Pope [51]; Tennekes [67]; Davidson [13]; Mcdonough [32] for an comprehensive introduction on the topic and to Saarenrinne and Piirto [55]; Lavoie et al. [28]; Segalini et al. [60]; Scharnowski et al. [56]; Ayegba and Edomwonyi-Otu [4]; Wang et al. [71] for a discussion on the impact of measurement resolution on the main turbulence variables. This section recalls the definitions that are required to solve the provided exercises.
In the classic Reynolds decomposition of stationary turbulent flows, the velocity field is decomposed into the sum of an average and a fluctuating part. For the provided 2D dataset, we write these as:
where denotes a Cartesian grid, is the mean-field and is the fluctuating field. The averaging in this formulation is an ensemble averaging. However, under the assumption of stationarity and ergodicity, this can be safely replaced by a more convenient time averaging.
The intensity of the fluctuation part is measured in terms of standard deviation or root mean square velocity, with components , . A second-order
A note on Turbulence
description is provided by the time correlation matrix of the fluctuating components7, which is a 2D flow reads:
This matrix varies at every location and in a statistical treatment of turbulence is linked to the Reynolds stress tensor , namely the additional stresses used to model the non-linear diffusion effect produced by turbulence on the mean flow. The trace of this matrix is linked to the turbulence kinetic energy , while the ratios and define the turbulence intensity vector .
The Reynolds stress tensors provides information on the degree of anisotropy of turbulence: in isotropic turbulence, one has and ; hence the matrix in (20) is diagonal. The eigenvalues of this matrix can be used to study the principal direction of turbulence momentum exchange [16, 61] using graphical representation such as the popular Lumley’s triangle [31]. This is essentially a principal component decomposition of the velocity components in a point.
Similarly, the covariance between gradients of the fluctuating components is linked to the turbulent dissipation rates . In its complete 3D form, this is defined as , with the kinematic viscosity, having used the indices to index the velocity components and axes and having implied summation. The full expression counts 12 terms and it is of difficult evaluation in a 2D measurement because of the missing information in the direction normal to the measurement plane (see Saarenrinne and Piirto [55]; Wang et al. [71] for an interesting discussion). This quantity measures the rate at which turbulence energy is converted into thermal energy by viscous dissipation and is essential in the modeling of turbulent flows. Because of the strong simplifications required by its computation, and because of the presence of gradients (which need extra care!), the exercise does not ask to compute this quantity8.
Finally, an important statistical quantity is the integral time scale . For the fluctuating component , having normalized temporal autocorrelation , this is defined (see also Oliveira et al. [44]) as
This integral converges (and the definition makes sense) only if the autocorrelation function tends to zero sufficiently fast. When this occurs, the integral time scale measures the time after which the random process becomes uncorrelated with itself or, in a more pictorial interpretation, the time within which the variable ‘remembers’ its history.
The reader is now ready for the second exercise.
Exercise 2: The statistics of a turbulent Flow
Study the statistics of the provided dataset, focusing on . (1) Compute the mean-field and the turbulence intensity field. (2) Analyze the convergence of these quantities in probe P1 and probe P2. (3) Compute the time autocorrelations in P1 and P2. If meaningful, compute the integral time scale. Test if the use of linear or cyclic operators makes any difference. Finally, (4) compute the Reynolds stress tensor at both locations.
Solution. We begin with a minor modification to code listing 1, where the velocity magnitude was sampled in the provided probes. We now store the full snapshots into a snapshot matrix U containing all fields of and a snapshot matrix V containing all fields of . This matrix will be important for the second part of the lecture. We also extract both velocity components in the probes P1,P2,P3. Here’s how the loop should be modified (see Exercise_2.py):
# Initialize the probes
P1_U=np.zeros(n_t,dtype="float32")
P1_V=np.zeros(n_t,dtype="float32")
P2_U=np.zeros(n_t,dtype="float32")
P2_V=np.zeros(n_t,dtype="float32")
P3_U=np.zeros(n_t,dtype="float32")
P3_V=np.zeros(n_t,dtype="float32")
# Prepare the snapshot matrices U and V
D_U=np.zeros((n_s//2,n_t),dtype="float32")
D_V=np.zeros((n_s//2,n_t),dtype="float32")
for k in range(0,n_t):
# Name of the file to read
Name=FOLDER+os.sep+'Res%05d'%(k+1)+'.dat'
# Read data from a file
# Here we have the two columns
DATA = np.genfromtxt(Name,usecols=np.arange(0,2),\
max_rows=nxny+1)
Dat=DATA[1:,:] # Remove the first raw with the header
# Get U component and reshape as grid
V_X=Dat[:,0]; V_X_g=np.reshape(V_X,(n_y,n_x))
# Get V component and reshape as grid
V_Y=Dat[:,1]; V_Y_g=np.reshape(V_Y,(n_y,n_x))
# Assign the vector like snapshots to snapshot matrices
D_U[:,k]=V_X; D_V[:,k]=V_Y
# Sample the U components
P1_U[k]=V_X_g[i1,j1]; P2_U[k]=V_X_g[i2,j2]; P3_U[k]=V_X_g[i3,j3];
# Sample the V Components
P1_V[k]=V_Y_g[i1,j1]; P2_V[k]=V_Y_g[i2,j2]; P3_V[k]=V_Y_g[i3,j3];
print('Loading Step '+str(k+1)+'/'+str(n_t))
Both the snapshot matrices and the velocity components sampled in probes are saved in a npz file for the exercises in the second part. Moreover, to limit memory usage, we work with variables in float32 rather than float64; this is accurate enough for our purposes and let the exercise comply with the limited memory of the author’s laptop.
Using the snapshot matrices, the mean flow and the root mean square velocity components can be computed in one line each:
A note on Turbulence
# U Component
mu_U=np.mean(D_U,axis=1) # Ensemble Mean
sigma_U=np.std(D_U,axis=1) # Ensemble STD
# V component
mu_V=np.mean(D_V,axis=1) # Ensemble Mean
sigma_V=np.std(D_V,axis=1) # Ensemble STD
Plotting these fields is now only a matter of reshaping the results back onto a matrix and using the provided functions (see the python code). The mean flow and the turbulence intensity are shown in Figure 6. Note that the latter is computed by normalizing the root mean square component with respect to the free stream velocity and not the local mean velocity. Most of the turbulence is localized in the wake region, where the fluctuation amplitude reaches up to 50% of the free stream velocity. This answers question (1).

Figure 6. Fig (a): Mean flow of the provided dataset in the first 1s (3000 snapshots) (b): Turbulence intensity, normalizing the root mean square field by the free stream.
The analysis of the convergence of the statistics at the probe locations P1 and P3 is similar to what was performed in the previous exercise. This is left as an exercise for the reader. The results are shown in Figure 7. The convergence in P1 is particularly fast, as the root mean square velocity is low and the mean velocity is large. The same is not true in P3, where convergence is barely achieved. The reader should analyze these statistics alongside the histogram in Figure 2: the s of acquisition is associated with the distribution centred at approximately 12 m/s.

Figure 7.
The reader is referred to Bruun [11] (chapter 12) for a discussion on the convergence of the mean and standard deviation as a function of the turbulence intensity.
Figure 7: Fig (a): Mean flow of the provided dataset in the first (3000 snapshots) (b): Turbulence intensity, normalizing the root mean square field by the free stream.
For the scope of this lecture, we can declare point (2) answered, although the reader is encouraged to proceed with further analysis9.
The time autocorrelations in P1 and P2 can be computed with the functions provided in the previous sections. Because the time series under analysis are rather long (3000 samples), the difference between circular and linear autocorrelation is not appreciable for lags ; the reader is nevertheless encouraged to explore the difference at larger lags. The results for P1 and P2, using circular autocorrelation, are shown in Figure 3.3 for the first . In both probes, oscillations of the flow are visible, although these are much stronger in P3 than in P1. As a result, none of the autocorrelations operators converge to zero, meaning that the signals are not purely ‘stochastics’ and a deterministic component could (and should!) be extracted before an integral time scale is to be computed10. Interestingly, the oscillations appear in phase, as if the entire flow field is locked at a given frequency.
Figure 8. Fig (a): Mean flow of the provided dataset in the first (3000 snapshots) (b): Turbulence intensity, normalizing the root mean square field by the free stream.
Finally, we close with the Reynolds stress tensor. This is essentially a covariance matrix. Arranging the velocity information in matrices of dimention , this is simply:
1 u1_f = P1_U - np.mean(P1_U); v1_f = P1_V - np.mean(P1_V)
2 U_f_1 = np.vstack((u1_f, v1_f)).T
3 Re_1 = 1/len(P1_U)*U_f_1.T.dot(U_f_1)
The results in points P1 and P2 are:
The scatter plot of the fluctuation components are shown in Figure 9. It is clear that the turbulent kinetic energy and the turbulent intensity is much larger in P3 than P1, and hence the distribution is broader in the first. In both points, however, the scatter plot shows a rather uncorrelated distribution of fluctuation, resulting in a nearly diagonal covariance: the flow is almost isotropic. ‘Almost’, because is slightly larger than in both points.

Figure 9. Scatter plot of the fluctuating components and at the location of probe P1 (Fig a) and probe P2 (Fig b).
Power Spectral Density and Coherence
We now move to the frequency representation of stochastic signals. This is slightly more involved than the frequency representation of deterministic signals. In a continuous domain, a stochastic signal is seldomly square-integrable, and it thus seldomly admits an ordinary Fourier transform. In the signal processing terminology, the definition of an appropriate frequency domain requires shifting the treatment from the notion of energy to the notion of power11: stochastic signals have generally infinite energy but finite power (Oppenheim, [45]; Hayes [19]).
Therefore, we shall not focus on the Fourier transform of a signal but at the Fourier transform of its autocorrelation (which is square integrable). The following transform exist for all signals of interest:
where is the pulsation and is the frequency and is the continuous Fourier transform of the autocorrelation function and is known as power spectral density. Note that this is real and even in because the time autocorrelation is symmetric (). The Fourier pairs of interest are thus
These are also known as Wiener-Khinchin relations. The notion of power spectral density is important because it allows to give a spectral representation to stochastic signals and thus to generalize the theory of linear time-invariant (LTI) systems (see Mendez [35]). This theory brings powerful and simple tools for system identification, filtering and forecasting, and we show some of them in practice in the next exercise (see Oppenheim [45]; Smith [63] for more).
Concretely, we are interested in the notion of coherency in relation to the frequency content of signals. The question motivating this section is simple: is there any chance to predict what is happening in one probe (e.g. P2), at least within a certain range of frequencies, given the signal we have in another probe (e.g. P3)?
We first need some definitions. Let us assume that a stochastic signal is the input of a linear and deterministic system which responds with a second stochastic signal . Uncorrelated noise might be added to the output of this system and we only see the resulting . If the power of the noise is too large, we might not be able to recover any reasonable estimate of and we will say that there is a poor level of coherency between and . Conversely, we might be able to identify an approximation of the underlying linear linking input and output. This link is here to be analyzed frequency by frequency.
Considering finite duration signals, the convergence problems of the Fourier representation are less stringent, and we can waive some of the formalism required for the continuous world: under the assumption of circular extension of the signals12, every digital signal has a Discrete Fourier Transform (DFT). Let and denote the DFT of the input and the output (see (17) for the definitions).
If a linear time invariant system links to , the output can be computed via convolution of the input with the system’s impulse response. From the convolution theorem, we know that in frequency domain this is a multiplication with the transfer function of the system , i.e. the DFT of the impulse response. We thus have:
We now introduce the discrete equivalent of (25). It is possible to show (see Oppenheim et al. [46] and (18)) that the power spectral density of the input is
Similarly, the power spectral density of the output is and we can additionally define the cross-spectral density as .
We can now craft a spectral function which measures how well the spectrum of the output correlates with the spectrum of the input. This is the coherence function:
At those frequencies for which (i.e. for which a LTI system could model the input-output relation), one has
Moreover, noticing that , we recover . At frequencies for which no LTI system can link the two spectra, the coherence is much close to zero. The function is undefined at ’s for which either or is zero.
To conclude this section, it is worth noticing that working with one single spectrum for both input and outputs makes little sense: an ensemble of time series leads to an ensemble of spectra. The classic approach to evaluating a stochastic signal’s frequency representation involves averaging, which could be either in the ensemble domain or in the time domain. Under the assumption of ergodicity, the second is usually preferred, and the result is the well known Welch’s method [73] or periodogram method. The computation is performed by dividing the signal into successive (and overlapping) blocks, computing the DFT of these, and then averaging the results. In practice, a smoothing window multiplies the signal in the time domain to cope with the problems arising by the (usually violated) assumption of periodicity. For , for instance, we have:
Here is the DFT of the windowed block , with , the window function designed to taper gracefully to zero at both endpoints of the block, and with denoting the number of blocks. Because the multiplication in the time domain is a convolution in the frequency domain, the method is essentially a smoothing operation on the signal’s spectrum; this is why the method is sometimes also called ‘smoothed spectrum’ and implemented in the frequency domain.
The Python’s scipy package offers robust functions to compute both the power spectral densities and the spectral coherence. We use both in the next exercise.
Exercise 3: Power Spectral Densities and Coherency The power spectral density of the signal in the probes P1 and P3 was shown in Figure 2. As in the previous exercise, let us consider only the first of
acquisition. First, practice with the notion of coherency by testing it on a synthetic signal . To construct this signal, filter the signal in P2 with a moving average filter of order three, i.e., impulse response , and add noise to the result. Next, show the original and synthetic signal’s power spectral densities and study the coherence function between the two signals. (1) What do you get and why? Finally, (2) study the coherency between the signals in P2 and P3.
Solution. We assign the signal to the input variable and perform the filtering plus noise superposition in the following lines:
1 data = np.load(’Sampled_PROBES.npz’)
2 n_T=3000; Fs=3000; dt=1/Fs
3 P2_U=data[’P2_U’][0:n_T];
4 P3_U=data[’P3_U’][0:n_T];
5 # Prepare the time axis
6 t=np.linspace(0,dt*(n_T-1),n_T)
7 # Define impulse response and ’input’
8 h=np.ones(3)/3; x=P2_U
9 # We padd to use the frequency domain:
10 h=np.concatenate([h,np.zeros(len(x)-3)])
11 # The transfer function will be
12 H=np.fft.fft(h)
13 # This is circular convolution
14 X=np.fft.fft(x)
15 # Add noise
16 N=np.random.normal(0,0.5*np.std(x),len(x))
17 y=np.real(np.fft.ifft(X*H))+N
We compute the power spectral densities and as follows:
1 N_P=300
2 f,Sxx_den = signal.welch(x, Fs, nperseg=N_P, scaling=’density’)
3 f,Syy_den = signal.welch(y, Fs, nperseg=N_P, scaling=’density’)
The reader should consult the documentation of the welch function. This function performs mean removal by default, while the scaling option ’density’ makes the output a density function (i.e. units are ‘V²/Hz’, with V the input units). The blocks for the averaging are here shown to be of size 300 while the overlapping is set by default to 50%. The power spectral densities of the original and the manipulated signal is shown in Figure 10a). The spectra are quite similar at low frequencies and feature the same peak at 459Hz. However, they become increasingly different at higher frequencies.

Figure 10. Fig (a): Power spectral densities of the original signal (stream-wise velocity component in P1) together with its manipulated version (b):cross-coherency between the signal and its manipulated version, together with the magnitude of the transfer function used in the manipulation.
We now compute the coherence function together with the transfer function of the moving average filter here implemented and plotting the results in the same graph:
1 # Prepare the transfer function of h
2 # curiosity: this is a nice tool for transf functions
3 wz, Hz = signal.freqz(h)
4 # Create the figure axis
5 fig, ax = plt.subplots(figsize=(6, 3))
6 # Compute the coherency
7 f, Cxy = signal.coherence(x, y, Fs, nperseg=N_P)
# Plot the results in decibels
plt.plot(f, 20*np.log10(Cxy),label='$C_{x,y}$')
# Plot the transfer function
plt.plot(wz/np.pi*Fs/2,20*np.log10(Hz), 'r',label='$|H(f)|$')
The result is in Figure 10b). It is clear that the coherency decreases following the same trend as the transfer function of the LTI system implemented: frequencies that are almost removed from the original signal leave the stage to the added random (incoherent) noise. The reader should repeat the exercise without adding noise.
Finally, the cross coherency between P2 and P3 is shown in Figure 3.4. The overall level of coherence is low, except for the regions at Hz. This corresponds to the frequency of the vortex shedding, characterized by coherent structures. We look for these in the next section.
Figure 11. Cross-coherency between the stream-wise velocity in P2 and P3. At approximately
Classic Modal Analysis
In the last exercise, we observed a strong correlation between the signals in P2 and P3 at the frequency of the vortex shedding. In the previous section, we have considered time series arising by sampling a specific point (and we took care of including a subscript in all the defined quantities).
In this section, we look for the statistics, the coherence and the frequency content of the flow field as a whole: the time series of interest is no longer a scalar, but a vector of dimension for the components and , and dimension if we take both components in the same ‘snapshot’.
More specifically, let denote the TR-PIV field over a uniform grid , with a matrix linear index, and a temporal discretization with the time step and the sampling frequency. In what follows, bold fonts are used to indicate matrices.
The goal of modal analysis is to describe the data as a linear combination of elementary information referred to as modes. Each mode has its own spatial structure and temporal evolution and can potentially describe essential features in the data. Each mode has a spatial structure , a temporal structure , and an amplitude that weights its importance. The sampled velocity field is thus expanded as
where the contribution of each mode is defined as and the spatial structures are obtained via normalization .
Assuming that all modes are (at least!) linearly independent, the number () of modes non-null amplitudes defines the dimensionality of the data for a given basis. Truncating the summation to modes provides an approximation of order with respect to a specific decomposition.
These tools were initially developed for identifying coherent structures in turbulent flows ([30, 8, 62]) and to construct reduced-order models ([21, 20, 7]) of fluid flows. Their range of application is nowadays much broader and include (among others) filtering ([53]), image processing ([33]) and validation of numerical simulations ([17]). By removing ‘irrelevant’ modes, we use the decomposition as a filter, and we compress data. By focusing on the dynamics described by ‘a few’ modes, we use decompositions as tools for constructing reduced-order models, which can significantly reduce the computational burdens in the simulations of large dynamical systems. A recent review of dimensionality reduction is proposed by Ahmed et al. [2].
In this section, we consider three decompositions in the form of eq. (30). These are the Discrete Fourier Transform (DFT) in subsection 4.1, the Proper Orthogonal Decomposition (POD) in subsection 4.2 and the Dynamic Mode Decomposition (DMD) in subsection 4.3. These can be distinguished by the choice of the temporal structure (see also Mendez [34]). We will not go into much implementation and algorithmic details (these are extensively treated in Mendez [34]) and we will use the Python package MODULO (see Ninni and Mendez [41])
In what follows, we use the subscripts , , and to denote, respectively, the modes linked to the DFT, the DMD and the POD.
The DFT
The DFT Modes
In the DFT, the temporal basis is the Fourier basis, which we have already encountered in (17): that is where , with , the frequency resolution and the sampling frequency.
The only difference with respect to the previous section (and the classic implementations in Python or Matlab) is that we will now normalize the DFT by and not . This normalization makes the basis orthonormal: if we construct a vector , for any of the frequencies we will have .
We now compute the DFT at every location and arrange the results in the form of spatial modes. From a practical point of view, we can leverage the organization of the data into a snapshot matrix and compute the DFT at every row. Again, this will be very convenient and make the decomposition accessible in just one line of code.
Let us consider the DFT of the signal at one specific location :
This operation can be interpreted in at least two ways. From a statistical point of view, this is a correlation (in time) between the temporal evolution of the signal at a certain position and a certain basis element . From a linear algebra point of view, this is a projection (or ‘inner product’) of the temporal evolution of the signal on to the basis element (see Mendez [35]).
The result of this operation is a set of coefficients (for the and the components) which is in essense a velocity field for every frequency . This field represents the Fourier spectra of the time series of and the at that specific point and pulsating at the single frequency . For each of these fields, we can compute the norm (the space domain) as and the normalized result will be denoted as . The set of gives the spatial structure (basis) of the DFT modes.
The decomposition derived can be arranged as a matrix factorization of the snapshot matrix , in which each column contains the snapshot properly reshaped. For the DFT, this will be:
where collects the spatial structures along its columns, is a diagonal matrix collecting the spectra and is the Fourier matrix collecting the temporal structures (note that this matrix is symmetric, i.e. ). Let us compute these for the provided dataset.
Exercise 4: The DFT Modes of the TR-PIV data
Consider the dataset in the first s. Using the algorithm, compute the DFT modes. Plot the spectra and the spatial structure of the modes with , namely the frequency of the vortex shedding.
Solution. We have the dataset already arranged in the form of snapshot matrices from the previous exercise. We load these and stack into a single snapshot matrix, to which we remove the time average:
# Load the snapshot matrices from Exercise 2
data = np.load(’Snapshot_Matrices.npz’)
Xg=data[’Xg’]; Yg=data[’Yg’]
D_U=data[’D_U’]; D_V=data[’D_V’]
# Assembly both components in one single matrix
D_M=np.concatenate([D_U,D_V],axis=0) # Reshape and assign
# We remove the time average (for plotting purposes)
D_MEAN=np.mean(D_M,1) # Temporal average (along the columns)
D=D_M-np.array([D_MEAN,]*n_t).transpose() # Mean Removed
The computation of the DFT fields can be done in one line of code:
PHI_SIGMA = (np.fft.fft(D, n_t, 1)) / (n_t ** 0.5)
Note that this is the product in (32) (hence the name). The remaining steps are (1) a normalization of all the columns, storing the results of the amplitudes in a vector:
# Initialize the PHI_F MATRIX (spatial structures)
PHI_F = np.zeros((D.shape[0], n_t), dtype=complex)
# Initialize the SIGMA_F vector (DFT Spectra)
SIGMA_F = np.zeros(n_t)
# Normalization loop
for r in tqdm(range(0, n_t)):
SIGMA_F[r] = abs(np.vdot(PHI_SIGMA[:, r], PHI_SIGMA[:, r]))** 0.5
PHI_F[:, r] = PHI_SIGMA[:, r] / SIGMA_F[r]
and (2) the sorting of the modes in decreasing order of amplitude:
# find indices for sorting in decreasing order
Indices = np.flipud(np.argsort(SIGMA_F))
# Sort all the sigmas
Sorted_Sigmas = SIGMA_F[Indices]
# Sort all the frequencies accordingly.
Sorted_Freqs = Freqs[Indices]
# Sorted Spatial Structures Matrix
Phi_F = PHI_F[:, Indices]
# Sorted Amplitude Matrix
SIGMA_F = np.diag(Sorted_Sigmas)
where the vector ‘Freqs’ is the one computed using scipy’s function
Freqs = np.fft.fftfreq(n_t) * Fs # Compute the frequency bins
This is essentially defined in the previous section. The resulting spectra is plotted in Figure 12a) versus the associated frequencies and in Figure 12b) in terms of the mode index . The first plot shows that, as expected, the leading modes are the ones at the frequency of the vortex shedding. The second plot shows that the ‘decay rate’ of the amplitude is modest, and many modes are needed to approximate this dataset despite its strong oscillatory nature. We conclude that despite the periodicity in the data, the convergence of the DFT is not particularly strong.

Figure 12. Fig (a): Amplitudes of the DFT modes as a function of the associated frequency . The leading modes are close to the vortex shedding frequency. Fig (b): amplitude of the DFT modes , in decreasing order, as a function of the index .
The spatial structures can then be shown after a reshaping. The leading structure (see entry Sorted_Freqs[0] has Hz) and the reader should note that modes come in pairs (only the postive portion of the spectra is here shown, from to ). The real and the imaginary part of the leading DFT modes can be reshaped as follows:
r =0;
# Take the real part of mode r
Phi_F_Mode_U_r = np.real(Phi_F[0:nxny,r]).reshape((n_x,n_y))
Phi_F_Mode_V_r = np.real(Phi_F[nxny::,r]).reshape((n_x,n_y))
# Take the imaginary part of mode r
Phi_F_Mode_U_i = np.imag(Phi_F[0:nxny,r]).reshape((n_x,n_y))
Phi_F_Mode_V_i = np.imag(Phi_F[nxny::,r]).reshape((n_x,n_y))
These fields are shown in Figure 13. Their superposition describes the travelling of vortices whose size is comparable to the cylinder. The two modes provide a simple kinematic approximation of the vortex shedding mechanism and explain why the signals in probe P2 and P3 have a strong level of coherency at Hz: both probes are located in regions where the first mode is ‘strongly present’.

Figure 13. Real (a) and imaginary (b) parts of the spatial structure of the leading DFT mode, capturing the vortex shedding.
While this exercise nicely showcased the strengths of the DFT; a simple variant can quickly showcase its limitations: the reader is invited to repeat the same exercise taking now all the snapshots of the dataset. The convergence decreases significantly, and two dominant frequencies will now appear (linked to the vortex shedding in the two stationary conditions, as described in § 3). These will have their associated modes, but the DFT will leave an important question open: when are these modes active?
An extension of tools from time-frequency analysis (e.g. Wavelets, [15]) to the ‘modal’ formalism could address these points but does not always improve the convergence and keeps the decomposition ’general’, i.e. with bases not adapted to the dataset at hand. The POD introduced in the next session gives the best possible convergence and tailors its basis to the data.
The POD
The previous exercise has highlighted the frequency localization capabilities of the DFT, but also its (generally) poor convergence. The DFT does not qualify as a ‘data-driven’ decomposition because the basis employed is pre-defined: in the DFT, we use the Fourier basis regardless of the data at hand.
We now enter into data-driven decompositions and thus seek to tailor the basis to the data. The Proper Orthogonal Decomposition (POD) is the oldest technique and arguably the most robust and simple tool. The decomposition is known with different names in different fields, and the reader is referred to Dawson [14] for a review.
The POD is the fundamental ‘energy-based’ decomposition: it provides the modes with the highest amplitude, and such that the convergence of an approximation with increasing number of modes is optimal. Identifying the temporal structures satisfying this condition is a constrained optimization problem (see (see Holmes et al. (1996) or Bishop (2006)) in the framework of Principal Component Analysis, PCA) and we here give the solution without proofs: the POD’s temporal structures must be eigenvectors of the temporal correlation matrix , with the expectation operator in the space domain. Hence, we must have:
having considered each temporal structure as a column vector, and having recalled, in the last summation, that the correlation matrix is symmetric and positive definite. The are, therefore, orthonormal like the in the DFT. What gives the POD the special status of ‘proper’ orthogonality, however, is that this specific choice of the temporal structure also makes the associated spacial structures orthonormal.
In addition to optimal convergence, this ‘proper orthogonality’ brings two interesting by-products (see also [37, 34]. The first is that the amplitudes of the modes are and thus the normalization step in the calculation of the spatial structures in (30) is no longer necessary. Then, it is possible to compute the spatial structures via correlation:
The second by-product is that the spatial structures are eigenvectors of the spatial correlation matrix , with the inner product in the time domain. Their orthogonality allows also to write:
Therefore, one might decide to work with the diagonalization of or depending on whether one has or , and compute by correlation the remaining structures (either the ’s via (34) or the via (35)).
Once completed, the arranging the POD in the matrix formalism as in (32) becomes:
For datasets uniformly spaced in time and space, this is a Singular Value Decomposition (SVD) of . We see this decomposition in action in the next exercise.
Exercise 5: The POD Modes of the TR-PIV data
Compute the POD of the full dataset (i.e. , corresponding to ). Show the amplitude of the modes, their spatial structures and the frequency content of their temporal structures.
Solution. We should re-run the script from Exercise 2 and build the full dataset of snapshot. Assuming that this is stored in a matrix as before, the first step is to compute the temporal correlation matrix. This is simply . However, because the computational cost of this operation hits the limit of the author’s laptop, we here use the memory saving feature of the MODULO package ([41]), recently implemented in Python by Schena and Mendez [57]. The following code loads the data and computes the snapshot matrix:
# Load the snapshot matrices from Exercise 2
data = np.load('Snapshot_Matrices.npz')
Xg=data['Xg']; Yg=data['Yg']
D_U=data['D_U']; D_V=data['D_V']
# Assembly both components in one single matrix
D_M=np.concatenate([D_U,D_V], axis=0) # Reshape and assign
# We remove the time average (for plotting purposes)
D_MEAN=np.mean(D_M,1) # Temporal average (along the columns)
D=D_M-np.array([D_MEAN,]*n_t).transpose() # Mean Removed
To use the MODULO package, we first create a MODULO object with some important information, then run the POD computation in one line:
m = MODULO(data=D,
n_Modes = n_r,
MEMORY_SAVING=True,
N_PARTITIONS=11)
# Prepare (partition) the dataset
D = m._data_processing() # Note that this will cancel D!
# Compute the mPOD
Phi_P, Psi_P, Sigma_P = m.compute_POD()
The first line in the script prepares the object declaring that the data to decompose is in the matrix , that we look for the first 500 modes, and that the memory saving feature will be requested with 11 partitions (Ninni and Mendez, [41]). This means that the code will break the dataset into 11 chunks and store it in temporary files, releasing some memory. Then, during the computation of the correlation matrix, the code loads and use one piece at a time. This allows computing the temporal correlation matrix saving considerable computational resources. The partitioning is created as soon as the second line in the script is run. Finally, the last line contains the computation of the POD.
The rest of the code is an exercise in reshaping and plotting data. Figure 4.2 shows the plot of the POD mode amplitude from the diagonal of Sigma_P. Interestingly, the amplitude of all modes tends to zero for : the first three modes contains most of the information in the data.
Figure 14. Amplitude decay of the POD Modes for the full dataset.
As in the previous exercise, the spatial structures are first reshaped onto the PIV grid and then plotted as fields. Figure 15 collects the spatial structures and the spectra of the first three leading modes. These modes are extensively discussed in Mendez et al. [38]. Briefly, the first mode describes the large scale variation of the flow, while the other two represent the vortex shedding in its entire evolution. Because POD modes are real, travelling patterns can only be described by POD pairs whose temporal structures are in quadrature.

Figure 15. Spatial structures of the first three POD modes (a) and spectra of the associated temporal structures (b).
The compression achievable by the POD is interesting for many purposes. From a ‘dynamical system’ perspective, the dataset can be seen as a discrete system in , with spatial points (2130 per components). Predicting the flow field a time requires predicting numbers. If one approximates the dataset with respect to the (reduced) POD basis:
the flow field at time is prescribed by 3 numbers, namely the value taken by the temporal structures . The three temporal coefficients evolve according to a dynamical system in and the spatial structures in (37) map (linearly!) this small dimensional system to the original one, producing an error which can be shown to be equal to in a sense [14, 35].
Figure 4.2 shows the trajectory of the reduced dynamical system in , which is simply the plot of the three temporal structures. In stationary conditions, the modes 2 and 3 describe a circular orbit whose diameter is larger at larger velocities. The transient phase moves the system state from the large to the smaller orbits along the axis of the first mode. At all times, the system stays on a paraboloid. The existence of this paraboloid has been derived analytically at low Reynolds numbers (, see Noack et al. [42]); this exercise shows that this manifold continues to exists also at much higher and during transient states.
Figure 16. Evolution of the reduced order dynamical system in the POD basis.
The DMD
The DFT modes have one frequency each, but this frequency is constrained to be an integer multiple of the fundamental frequency with the observation time. This potentially generates convergence issues due to windowing problems. The POD modes have the best possible convergence, but its modes are of difficult interpretation: all the information is extremely compressed into a few modes.
The Dynamic Mode Decomposition (DMD) solves potential windowing problems while putting the “frequency-based” formalism of the DFT into the “data-driven” or “tailored-basis” formalism of the POD. The idea (introduced by Schmid [59]) is to fit a linear dynamical system onto a reduced dynamical system derived via POD. A different formulation was proposed by Rowley et al. [54] and the reader is referred to Chen et al. [12] for an interesting discussion on the link between DMD and DFT.
The idea of fitting a linear dynamical system to the POD reduced-order model of the data has a long history, and it is interesting to note that modern DMD algorithms are remarkably similar to what was introduced in the late ’80s as Principal Oscillation Patterns (POP, Hasselmann [18]; von Storch and Xu [70]) or Linear Inverse Modeling (LIM, Penland [49]; Penland and Magorian [50]) in climatology (see also Tu et al. (2014); Mendez et al. [38]; Uruba [69]).
The basic algorithm described in this section is the one implemented in MODULO and proposed by Tu et al. (2014), which is briefly described in this section. Let and be two portions of the snapshot matrix differing by a one shift in time. If a linear dynamical system ’moves’ to , then this system is characterized by a propagator matrix such that . The eigenvalues of this propagator governs the evolution of the system and its eigenvectors are the DMD modes we are looking for.
Solving for is a canonical least square problem, which could be solved via the pseudo-inverse of . This can be computed via POD (i.e. SVD) of :
However, computing the eigendecomposition of is a poorly conditioned and computationally prohibitive problem in most real applications. The solution is then to leverage on the compression achievable by POD. Projecting the whole problem onto a reduced POD basis , with , the reduced system in becomes:
where the approximation lies in the second step, i.e. in the assumption that with the identity matrix of size .
In this reduced system, the reduced propagator is thus:
having introduced (38). The eigenvalue decomposition of this propagator governs the evolution of the reduced system in (39). Structures linked to eigenvalues that have vanish at large times while those that have explode. Structures linked to are purely harmonic, like the DFT modes. However, their frequency is not constrained to be multiple of the fundamental tone. Finally, the (complex) spatial structure of the DMD can be computed by back-propagating in the original space using the POD basis: .
Exercise 6: The DMD Modes of the TR-PIV data As discussed in Mendez et al. [38], the DMD of the full dataset encounters serious convergence problems. After all, the trajectories in 4.2 cannot be approximated by a linear dynamical system. We thus focus again on the dataset in the first (that is ). Compute the DMD spectra and show the spatial structure of the dominant mode.
Solution. Using MODULO, the call to the DMD function is a one-line command, provided that the snapshot matrix has been prepared as in the previous exercises. The DMD with n_r=500 modes can be computed as follows:
#%% Perform the DMD with Modulo
# Initialize a ’MODULO Object’
m = MODULO(data=D,
MEMORY_SAVING=True,
n_Modes = n_r,
N_PARTITIONS=11)
# Prepare (partition) the dataset
D = m._data_processing() # Note that this will cancel D!
# Compute the POD
Phi_D, Lambda, freqs, a0s = m.compute_DMD()The output provides the spatial structures of the DMD (in Phi_D), the eigenvalues of the reduced propagator (Lambda), the frequency (in Hz) linked to each eigenvalue
and the initial amplitudes of all modes (a0’s). We can now plot the DMD spectra in the complex plane:
1 fig, ax = plt.subplots(figsize=(4, 4)) # This creates the figure
2 plt.plot(np.imag(Lambda),np.real(Lambda),'ko')
3 circle = plt.Circle((0,0),1,color='b', fill=False)
4 ax.add_patch(circle)
5 ax.set_aspect('equal')
and in terms of frequencies:
1 fig, ax = plt.subplots(figsize=(6, 4)) # This creates the figure
2 plt.plot(freqs,np.abs(Lambda),'ko')
The results are shown in Figure 17a and b) respectively. We note that all the computed eigenvalues have absolute values smaller than one and will thus vanish in the long run. Two of these have (see fig a): these are almost on the unitary circle in the complex domain and therefore are nearly harmonics. Interestingly, these correspond to , i.e. the frequency of the vortex shedding (see Figure b). The spatial structures are complex and can be plotted as done for the Fourier modes. The reader finds the scripts in the provided Exercise_6.py file. The spatial structures are nearly indistinguishable from Fourier modes.
In terms of convergence, the DMD does not converge. The temporal structures of this decomposition are not even orthogonal, despite the provided case exhibiting strong periodicity. For a more extensive discussion on the convergence problems of the algorithm, the reader is referred to Mendez et al. [37, 38]; Mendez [34].

(a)

(b)
Figure 17. Fig (a): DMD spectra for the TR-PIV dataset in , plotted in the complex plane together with the unitary circle. Fig (b): absolute values of the DMD eigenvalues as a function of the corresponding frequency in Hertz. The modes with correspond to the vortex shedding.
Beyond Classic Decompositions
The previous sections gave a brief overview of the ‘classic’ decompositions. We here briefly discuss two possible extensions.
The Multiscale Proper Orthogonal Decomposition
In the POD, the temporal structures are eigenvectors of the temporal correlation matrix. These have unconstrained frequency content derived under the energy optimality criterium. As a result (see exercise 4), this decomposition finds an effective representation of the data, but the spectra of each mode are broad. In the DMD, the temporal structures are eigenvectors of the reduced propagator which best approximate the dataset as a linear dynamical system. These have one (complex) frequency, i.e. with . As a result (see exercise 5), this decomposition finds the ‘optimal’ frequencies, but the resulting decomposition has convergence problems even in simple datasets.
The limits of these two ‘extreme approaches’ (i.e. ‘energy optimality at all costs’ or ’one-frequency per mode’ at all costs) have motivated many hybrid methods (see Mendez et al. [37]; Mendez [34] for a brief literature review).
The Multiscale Proper Orthogonal Decomposition, first proposed in Mendez et al. [36] aims at finding a compromise between the energy optimality of the POD and the spectral purity of the DMD. The idea is to introduce the notion of optimality within a prescribed range of frequencies. As extensively illustrated in Mendez et al. [37] and Mendez [34], the mPOD is essentially a combination of POD and Multi-resolution Analysis. The MRA can be conveniently performed on the temporal correlation matrix.
The first step of the mPOD is the partitioning of the temporal correlation matrix into the contribution of scales. This is done by a set of FIR filters with impulse response :
where denotes convolution and is correlation matrix for the scale . The mPOD temporal structures are eigenfunctions of the different contributions, i.e. . The filters in Eq. (41) are designed with two criteria: they have separable impulse response and have non-overlapping band-pass regions in the frequency domain. Then, given the filter’s transfer function
the product is identically zero for all .
The mPOD use Eq. (41) to approximate the correlation matrix using only filters with unitary transfer function along the diagonal, i.e. . Then, it is possible to show that the eigenvectors of all the correlation matrices are mutually orthogonal. Moreover, their frequency content is limited by the transfer function of the filter identifying the corresponding scale: this means that structures from different scales have no common frequencies. Given the Fourier transform of the r-th eigenvectors of , the product is identically zero for all and if .
The temporal basis collecting the eigenvectors of all scales is the mPOD basis. This is orthogonal by construction and spans ; it thus allows for a lossless decomposition while having modes that are optimal within the prescribed range of frequencies.
The associated spatial structures are finally computed via projection and normalization, as done for the DFT in (31) or the POD13 in (34):
with .
The software package MODULO provides an efficient implementation of the mPOD given a frequency splitting vector. We use it in the next exercise.
Exercise 7: The mPOD Modes of the TR-PIV data
Compute the mPOD of the full dataset (i.e. , corresponding to ). Consider the following frequency band (in Hz): F_V=np.array([10,290,320,430,470]).
In practice, we look for mPOD modes that have frequency content limited by the entries in F_V. Modes that have frequency content in the range 10-290 Hz should not have frequency content in the range 290-320 Hz, etc. This decomposition has six scales: 0-10, 10-290 … until 470-1500 Hz.
The goal is to identify the structures associated with the first stationary phase, hence a vortex shedding at Hz, from the ones associated with the second stationary phase, shedding at Hz. Moreover, we are interested in the large scale evolution of the flow, which we consider describable with modes having frequency Hz. This explains the choice of F_V.
In this exercise, the reader computes the spatial structures, the temporal structures and the Fourier spectra of the modes linked to the two shedding frequencies.
Solution. Assuming that the data is arranged in a snapshot matrix like in the previous exercises, the mPOD with the prescribed parameters is computed as follows
1 m = MODULO(data=D, MEMORY_SAVING=True, n_Modes = n_r,
2 N_PARTITIONS=11)
3 # Prepare (partition) the dataset
4 D = m._data_processing() # Note that this will cancel D!
5 # mPOD Settings
6 # Frequency splitting Vector
7 F_V=np.array([10,290,320,430,470])
8 # This vector collects the length of the filter kernels.
9 Nf=np.array([1201,801,801,801]);
10 # Decide which scales to keep
11 Keep=np.array([1,1,0,1,0])
12 # Others
13 Ex=1203; boundaries = ’nearest’; MODE = ’reduced’
14 Phis, Psis, Sigmas = m.compute_mPOD(Nf, Ex, F_V, Keep,15 boundaries, MODE, 1/Fs)
The first four lines are the same as for all decompositions in MODULO. Line 7 prescribes the frequency splitting vector proposed in the exercise. MODULO operates with FIR filter [19] and the vector collects the length of the associated impulse responses (which are computed by default using Hamming windows). Line 11 defines the ‘Keep’ vector: if 1, the corresponding scale is kept; if 0, it is skipped. By setting some entries of this vector to 0, the mPOD also acts as a filter in the frequency domain.
The last options, in line 13, prescribe the maximum extension of the signals () during the filtering and the necessary boundary extension for the filtering (the convolutions in (41)): in this case, the option ‘nearest’ pads the signals with the last or the first entry. Finally, the mode ’reduced’ controls whether the basis to export is a reduced one or a complete one. The results are stored as a matrix factorization like the POD, and hence the rest of the exercise is a plotting exercise no different from Exercise 5. We thus skip the coding and show the results in Figure 18.

Figure 18.
Figure 18 (PDF p. 40). Figure (a) and (b). Spatial structures of the two modes linked to the vortex shedding during the two stationary conditions. The temporal structures in (c) and (d) show that the modes are active within certain time intervals, corresponding to the stationary phases. The Fourier spectra of the temporal structures (e) and (f) show that these modes are also localized in the frequency domain.
As expected, the mPOD modes capture the modes for the shedding frequencies in the different time intervals. Interestingly, the mPOD inherits from its MRA architecture the ability to localize modes in the frequency domain (these have a relatively narrow frequency bandwidth) with the ability to localize modes in the time domain (these are active only within a specific time interval).
The Kernel PCA
In exercise 5, the reader experienced the remarkable compression capabilities of the POD. With only three modes, it is possible to build an approximation of a complex flow with an error of . This is the best possible result for any linear decomposition.
What if we move to a nonlinear framework? Nonlinear dimensionality reduction is a well-established subfield of machine learning (Alpaydin, [3]; Bishop, [9]), but its application to fluid mechanics is at its infancy (see Noack et al. [42]; Ahmed et al. [2]; Mendez et al. [39]). As for all the other sections, the field is too broad to be explored in this lecture, but a glance at the general ideas might be given on one page. Moreover, powerful libraries such as scikit-learn (Pedregosa et al., [48]) make many nonlinear decompositions accessible in just one line of code.
We consider the most popular nonlinear tool for dimensionality reduction: the Kernel Principal Component Analysis (kPCA) by Schölkopf et al. [58]. Instead of working with the snapshot matrix , we work with a kernelized version of it: .
Here is a kernel function which maps the snapshots into the so-called feature space . While the spatial structures of the POD (PCA in machine learning terminology) are eigenvectors of , the spatial structures of the kPCA (or kPOD, in a sense) are eigenvectors of . These are (orthonormal) nonlinear functions of the dataset and their evolution in time (the temporal structures ) can be computed by projecting the data onto this basis.
If we focus on the forward problem, i.e. the compression problem, we are looking for the projection of the datasets onto the spatial structures . If the kernel functions are well-chosen, these projections can be computed without knowing the 's (Bishop, [9])!
The trick to perform this computation is known as kernel trick (Bishop, [9]; Schölkopf et al., [58]) and goes as follows. Let us write the as linear combinations of features:
where is the vector collecting the coefficients of the combination. Introduce this ansatz into the eigenvalue problem and multiply by on both sides:
where collects the inner products in the feature space. If the kernel function is carefully chosen Bishop [9], the inner products defining these matrix can be computed via a kernel function:
where and are two columns of , i.e. two snapshots of the data. If this matrix is invertible, equation (45) reduces to , i.e. the eigendecomposition of . Given the eigenvectors , the projection of a feature vector onto the principal component in the feature space is:
Computing these temporal structures is the forward kPCA problem. Reconstructing the dataset from these (i.e. the inverse kPCA), on the other hand, require the computation of the and this is a complex nonlinear optimization problem. The reader is referred to Bakir et al. [6] for more information. In the next exercise, we close this section by showing how to compute the kPCA of the provided TR-PIV dataset.
Exercise 8: The kPCA Manifold of the TR-PIV data
Compute the best 3 dimensional manifold approximating the full dataset using forward kPCA. Then, use this manifold to retrieve back an approximation of the velocity field. What is the least square error of the approximation?
Solution. As usual, assuming that the data is arranged in a snapshot matrix as for the previous exercises, this exercise can be solved in one line of code. Scikit-learn provides a powerful function for kPCA:
1 D_s=D.T
2 from sklearn.decomposition import KernelPCA
3 kpca = KernelPCA(kernel="rbf", n_components=3,
4 fit_inverse_transform=True,
5 alpha=1,gamma=None)
6 Psi_K = kpca.fit_transform(D_s)The convention used by scikitlearn is that of having data realizations (in our case snapshots of the TR-PIV measurement) along the rows of the feature matrix; hence the transposition. The kPCA is here performed using Gaussian Radial Basis functions (see Bishop [9]; Mendez et al. [39]) as kernel functions, with shape factor taken as a default (i.e. in this case). The call to the function is performed in two steps: line 3 create the kpca object while line 5 runs the function ‘fit_transform’ within the object. The parameter is a regularization parameter used to compute the inversion (as described in Bakir et al. [6]). The plot of the 3D manifold derived by the kPCA is simply:
1 fig = plt.figure(figsize=(7,4))
2 ax = fig.add_subplot(111, projection='3d')
3 ax = Axes3D(fig, elev=-150, azim=48)
4 ax.dist = 135 ax.scatter(Psi_K[0:n_t:2,1],
6 -Psi_K[0:n_t:2,2],
7 Psi_K[0:n_t:2,0], 'ko')
This is shown in Figure 5.2. The plot of these structures is made to have the same orientation as the one obtained via POD.

Figure 19. Evolution of the reduced order dynamical system in the kPOD basis.
It is thus possible to appreciate that the manifold is axisymmetric and that the system’s orbits remain circular in both the stationary phases. However, the nonlinear mapping results in a different shape, with much narrower orbits at the highest velocity and much larger at the highest (see animations in the presentation). The transitory is also different. The most remarkable result is yet to come: the error obtained by mapping this manifold back to can be computed as follows:
1 X_rec_kpca = kpca.inverse_transform(Psi_K)
2 Error_kpca=np.linalg.norm(D_s-X_rec_kpca)/np.linalg.norm(D_s)
3 print(Error_kpca)
The reader should find the result in the console: 2.983e-5. This is practically zero and six orders of magnitudes smaller than the error produced by a POD approximation of the same dimension. Somehow, the manifold in Figure 5.2 contains the same amount of information available in the 4260 dimensional space where the TR-PIV velocity fields live and evolve. Isn’t this remarkable?
Final Remarks
These notes offered a tour de force into signal processing, modal analysis and dimensionality reduction to process spatiotemporally resolved data like the one produced by modern image velocimetry. The lecture opened with statistical tools for time series analysis (§ 4), moved to classic modal analysis (§ 5) and explored modern extensions such as multiscale data-driven decompositions and nonlinear (kernel-based) methods (§ 6).
While the topics in Section 3 are textbook material for students in fluid dynamics for at least 30 years, the topics in Section 4 have become standard tools in the researcher’s toolbox more recently. It is thus not surprising that all statistical tools presented in Section 3 have already been integrated into the theory and the modelling of fluid flows, while the same is not entirely true, yet, for the topics in Section 4. However, the trends are clear: today, these are popular tools to extract patterns, to construct simplified representations, or to filter experimental (and numerical) data. The reader is encouraged to dive into the provided exercises to gain computational proficiency on the subject. Finally, the topics in Section 4 are at the forefront of fluid dynamics research and the ever-growing intersection between signal processing and machine learning. The field is very active, and the provided notes and exercises give a good starting point.
References
- [1]Adrian, R. J. (2005). Twenty years of particle image velocimetry. Experiments in Fluids, 39(2):159–169.DOI
- [2]Ahmed, S. E., Pawar, S., San, O., Rasheed, A., Iliescu, T., and Noack, B. R. (2021). On closures for reduced order models—a spectrum of first-principle to machine-learned avenues. Physics of Fluids, 33(9):091301.DOI
- [3]Alpaydin, E. (2014). Introduction to Machine Learning. The MIT Press.
- [4]Ayegba, P. O. and Edomwonyi-Otu, L. C. (2020). Turbulence statistics and flow structure in fluid flow using particle image velocimetry technique: A review. Engineering Reports, 2(3).DOI
- [5]Baj, P., Bruce, P. J. K., and Buxton, O. R. H. (2015). The triple decomposition of a fluctuating velocity field in a multiscale flow. Physics of Fluids, 27(7):075104.
- [6]Bakir, G., Weston, J., and Bernhard, S. (1999). Learning to find pre-images. In Proceedings of the 16th International Conference on Neural Information Processing Systems.
- [7]Benner, P., Gugercin, S., and Willcox, K. (2015). A survey of projection-based model reduction methods for parametric dynamical systems. SIAM Review, 57(4):483–531.DOI
- [8]Berkooz, G., Holmes, P., and Lumley, J. L. (1993). The proper orthogonal decomposition in the analysis of turbulent flows. Annual Review of Fluid Mechanics, 25(1):539–575.
- [9]Bishop, C. M. (2006). Pattern Recognition and Machine Learning. Springer-Verlag New York Inc.
- [10]Borodin, A. N. and Salminen, P. (2002). 7. ornstein–uhlenbeck process. pages 528–570. Birkhäuser Basel.DOI
- [11]Bruun, H. H. (1995). Hot-Wire Anemometry: Principles and Signal Analysis. OXFORD UNIV PR.
- [12]Chen, K. K., Tu, J. H., and Rowley, C. W. (2012). Variants of dynamic mode decomposition: Boundary condition, koopman, and fourier analyses. Journal of Nonlinear Science, 22(6):887–915.DOI
- [13]Davidson, P. A. (2004). Turbulence : an introduction for scientists and engineers. Oxford University Press, Oxford, UK New York.
- [14]Dawson, S. (2020). Data Driven Fluid Mechanics: Combining First Principles and Machine Learning, chapter The Proper Orthogonal Decomposition. Cambridge University Press; Ed. Miguel Alfonso Mendez and Andrea Ianiro and Bernd Noack and Steve Brunton. Also available as online lecture from the VKI Lecture series ’Machine Learning for Fluid Mechanics, 2020’: https://www.youtube.com/watch?v=IcqBbtWIcIc.
- [15]Discetti, S. (2020). Data Driven Fluid Mechanics: Combining First Principles and Machine Learning, chapter Mathematical Tools, Part II: Time-Frequency Analysis. Cambridge University Press; Ed. Miguel Alfonso Mendez and Andrea Ianiro and Bernd Noack and Steve Brunton. Also available as online lecture from the VKI Lecture series ’Machine Learning for Fluid Mechanics, 2020’: https://www.youtube.com/watch?v=54d04zaxmQU.
- [16]Emory, M. and Iaccarino, G. (2014). Visualizing turbulence anisotropy in the spatial domain with componentality contours by.
- [17]Erik, M. K., Dalibor, C., and M., P. J. (2007). Pod as tool for comparison of piv and les data. In 7th International Symposium on Particle Image Velocimetry, Rome, Italy.
- [18]Hasselmann, K. (1988). PIPs and POPs: The reduction of complex dynamical systems using principal interaction and oscillation patterns. Journal of Geophysical Research, 93(D9):11015.DOI
- [19]Hayes, M. (2011). Schaums Online of Digital Signal Processing. McGraw-Hill Education - Europe.
- [20]Holmes, P., Lumley, J. L., and Berkooz, G. (1996). Turbulence, Coherent Structures, Dynamical Systems and Symmetry. Cambridge University Press.
- [21]Holmes, P. J., Lumley, J. L., Berkooz, G., Mattingly, J. C., and Wittenberg, R. W. (1997). Low-dimensional models of coherent structures in turbulence. Phys. Rep., 287(4):337–384.DOI
- [22]Hsu, H. (2013). Schaum’s Outline of Signals and Systems, 3rd Edition (Schaum’s Out-lines). McGraw-Hill Education.
- [23]Ianiro, A. (2020). Data Driven Fluid Mechanics: Combining First Principles and Machine Learning, chapter Good Practice and Applications of Data-Driven Modal Analysis. Cambridge University Press; Ed. Miguel Alfonso Mendez and Andrea Ianiro and Bernd Noack and Steve Brunton. Also available as online lecture from the VKI Lecture series ’Machine Learning for Fluid Mechanics, 2020’: https://www.youtube.com/watch?v=H6twKFTCd2k.
- [24]Johansson, R. (2018). Numerical Python. Apress.
- [26]Kay, S. M. (1993). Fundamentals of Statistical Processing, Volume I. Prentice Hall.
- [27]Langtangen, H. P. (2016). A Primer on Scientific Programming with Python. Springer-Verlag GmbH.
- [28]Lavoie, P., Avallone, G., Gregorio, F. D., Romano, G. P., and Antonia, R. A. (2007). Spatial resolution of PIV for the measurement of turbulence. Experiments in Fluids, 43(1):39–51.DOI
- [29]Ljung, L. and Glad, T. (1994). Modeling of Dynamic Systems. Prentice Hall.
- [30]Lumley, J. L. (1970). Stochastic Tools in Turbulence. DOVER PUBN INC.
- [31]Lumley, J. L. and Newman, G. R. (1977). The return to isotropy of homogeneous turbulence. Journal of Fluid Mechanics, 82(1):161–178.
- [32]Mcdonough, J. M. (2004). Introductory lectures on turbulence physics, mathematics and modeling.
- [33]Mendez, M., Raiola, M., Masullo, A., Discetti, S., Ianiro, A., Theunissen, R., and Buchlin, J.-M. (2017). POD-based background removal for particle image velocimetry. Experimental Thermal and Fluid Science, 80:181–192.DOI
- [34]Mendez, M. A. (2020a). Data Driven Fluid Mechanics: Combining First Principles and Machine Learning, chapter Generalized and Multiscale Data-Driven Modal Analysis. Cambridge University Press; Ed. Miguel Alfonso Mendez and Andrea Ianiro and Bernd Noack and Steve Brunton. Also available as online lecture from the VKI Lecture series ’Machine Learning for Fluid Mechanics, 2020’: https://www.youtube.com/watch?v=BJQzk03HaTk.
- [35]Mendez, M. A. (2020b). Data Driven Fluid Mechanics: Combining First Principles and Machine Learning, chapter Mathematical Tools, Part I: Continuous and Discrete LTI Systems. Cambridge University Press; Ed. Miguel Alfonso Mendez and Andrea Ianiro and Bernd Noack and Steve Brunton. Also available as online lecture from the VKI Lecture series ’Machine Learning for Fluid Mechanics, 2020’: https://www.youtube.com/watch?v=qvZmKr6fhW4.
- [36]Mendez, M. A., Balabane, M., and Buchlin, J.-M. (2018a). Multi-scale proper orthogonal decomposition (mPOD). In AIP Conference Proceedings.DOI
- [37]Mendez, M. A., Balabane, M., and Buchlin, J.-M. (2019a). Multi-scale proper orthogonal decomposition of complex fluid flows. Journal of Fluid Mechanics, 870:988–1036.DOI
- [38]Mendez, M. A., Hess, D., Watz, B. B., and Buchlin, J.-M. (2020). Multiscale proper orthogonal decomposition (mPOD) of TR-PIV data—a case study on stationary and transient cylinder wake flows. Measurement Science and Technology, 31(9):094014.DOI
- [39]Mendez, M. A., Pino, F., and Fiore, M. (2019b). Optimization Methods for Computational Fluid Dynamics, chapter Machine Learning for Fluid Mechanics: Challenges, Opportunities and Perspectives. von Karman Institute for Fluid Dynamics LS.
- [41]Ninni, D. and Mendez, M. A. (2020). MODULO: A software for multiscale proper orthogonal decomposition of data. Software X, 12:100622.arxiv.org/abs/2004.12123
- [42]Noack, B., Ehlert, A., Nayeri, C., and Morzynski, M. (2020). Data Driven Fluid Mechanics: Combining First Principles and Machine Learning, chapter Mathematical Tools, Part II: Time-Frequency Analysis. Cambridge University Press; Ed. Miguel Alfonso Mendez and Andrea Ianiro and Bernd Noack and Steve Brunton. Also available as online lecture from the VKI Lecture series ’Machine Learning for Fluid Mechanics, 2020’: https://www.youtube.com/watch?v=iehMMhDqm ys.
- [43]Øksendal, B. (1998). Stochastic Differential Equations. Springer Berlin Heidelberg.
- [44]Oliveira, T. F., Misserda, R. B., and Cunha, F. R. (2007). Dynamical simulation and statistical analysis of velocity fluctuations of a turbulent flow behind a cube. Mathematical Problems in Engineering, 2007:1–28.DOI
- [45]Oppenheim, A. V. (2015). Signals, Systems and Inference. Pearson Education.
- [46]Oppenheim, A. V., Willsky, A. S., and with S. Hamid (1996). Signals and Systems (2nd Edition). Pearson.
- [47]Paul J. Deitel, H. D. (2019). Intro to Python for Computer Science and Data Science: Learning to Program with Ai, Big Data and the Cloud. PEARSON.
- [48]Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830.arxiv.org/abs/1201.0490
- [49]Penland, C. (1996). A stochastic model of IndoPacific sea surface temperature anomalies. Physica D: Nonlinear Phenomena, 98(2-4):534–558.DOI
- [50]Penland, C. and Magorian, T. (1993). Prediction of niño 3 sea surface temperatures using linear inverse modeling. Journal of Climate, 6(6):1067–1076.doi.org/10.1175/1520-0442(1993)006<1067:ponsst>2.0.co;2
- [51]Pope, S. B. (2000). Turbulent Flows. Cambridge University Press.
- [52]Raffel, M., Willert, C. E., Scarano, F., Kähler, C. J., Wereley, S. T., and Kompenhans, J. (2018). Particle Image Velocimetry. Springer International Publishing.
- [53]Raiola, M., Discetti, S., and Ianiro, A. (2015). On PIV random error minimization with optimal POD-based low-order reconstruction. Experiments in Fluids, 56(4).DOI
- [54]Rowley, C., Mezić, I., Bagheri, S., Schlatter, P., and Henningson, D. (2009). Spectral analysis of nonlinear flows. J. Fluid Mech., 641:115.
- [55]Saarenrinne, P. and Piirto, M. (2000). Turbulent kinetic energy dissipation rate estimation from PIV velocity vector fields. Experiments in Fluids, 29(7):S300–S307.DOI
- [56]Scharnowski, S., Bross, M., and Kähler, C. J. (2018). Accurate turbulence level estimations using PIV/PTV. Experiments in Fluids, 60(1).DOI
- [57]Schena, L. and Mendez, M. A. (2020). A python package for modulo (modal multiscale pod). https://modulo.readthedocs.io/en/latest/intro.html. Accessed: 2021-11-10.
- [58]Schölkopf, B., Smola, A., and Müller, K.-R. (1997). Kernel principal component analysis. In Lecture Notes in Computer Science, pages 583–588. Springer Berlin Heidelberg.
- [59]Schmid, P. (2010). Dynamic mode decomposition of numerical and experimental data. J. Fluid Mech., 656:5–28.
- [60]Segalini, A., Bellani, G., Sardina, G., Brandt, L., and Variano, E. A. (2014). Corrections for one- and two-point statistics measured with coarse-resolution particle image velocimetry. Experiments in Fluids, 55(6).DOI
- [61]Simonsen, A. J. and Krogstad, P.-Å. (2005). Turbulent stress invariant analysis: Clarification of existing terminology. Physics of Fluids, 17(8):088103.DOI
- [62]Sirovich, L. (1987). Turbulence and the dynamics of coherent structures: Part i. coherent structures. Quart. Appl. Math., 45(3):561–571.DOI
- [63]Smith, J. O. (2007a). Introduction to Digital Filters: with Audio Applications. W3K Publishing.
- [64]Smith, J. O. (2007b). Mathematics of the Discrete Fourier Transform (DFT): with Audio Applications. W3K Publishing.
- [66]Svein Linge, H. P. L. (2019). Programming for Computations - Python. Springer International Publishing.
- [67]Tennekes, H. (1972). A first course in turbulence. The MIT Press, Cambridge, Massachusetts.
- [69]Uruba, V. (2012). Decomposition methods in turbulence research. EPJ Web of Conferences, 25:01095.DOI
- [70]von Storch, H. and Xu, J. (1990). Principal oscillation pattern analysis of the 30- to 60-day oscillation in the tropical troposphere. Climate Dynamics, 4(3):175–190.DOI
- [71]Wang, G., Yang, F., Wu, K., Ma, Y., Peng, C., Liu, T., and Wang, L.-P. (2021). Estimation of the dissipation rate of turbulent kinetic energy: A review. Chemical Engineering Science, 229:116133.DOI
- [72]Wang, R. (2009). Introduction to Orthogonal Transforms. Cambridge University Press.
- [73]Welch, P. (1967). The use of fast fourier transform for the estimation of power spectra: A method based on time averaging over short, modified periodograms. IEEE Transactions on Audio and Electroacoustics, 15(2):70–73.
- [74]Williamson, D. (1999). Discrete-time Signal Processing. Springer London.




