betaFeedback

WorksPhysics and Astronomy

High-order stabilized matrix-free simulation of rotating mixing devices using the Mortar Element Method

Abstract

We present a finite element framework to simulate rotating mixing devices using the Mortar Element Method as a domain decomposition strategy. The model is implemented within a matrix-free Navier-Stokes framework which uses a high-order Continuous Galerkin method. The discretized domain is subdivided into rotor and stator parts. An Arbitrary Lagrangian-Eulerian approach accounts for the relative rotor-stator motion, and stabilization is ensured through the Streamline-Upwind/Petrov-Galerkin and Pressure-Stabilizing Petrov-Galerkin methods. The rotor-stator domains are connected by an interface composed of mortar cells, and continuity is weakly enforced in a Discontinuous Galerkin fashion by accounting for boundary integrals at the rotor-stator interface. Verifications of the convergence order in two-dimensional steady and transient examples report optimal rates. The geometric non-conformity created at the mortar interface due to rotor rotation does not introduce significant error in the solution. A three-dimensional example is used to investigate the model's scalability, which yields ideal strong scaling for large problems. A two-dimensional Rushton impeller example uses a torque analysis to showcase the mesh convergence, and the corresponding velocity profile is in agreement with existing numerical results. In a three-dimensional pitched blade turbine case, the power number curve (Np vs Re) shows good agreement with experimental data for Reynolds number values from 1 to 2000. An energy balance analysis reports a numerical dissipation of 1% for Re=200 and of 10% for Re=2000. By exploiting modern hardware capabilities through matrix-free methods, the proposed model is a robust, accurate, and efficient framework suitable for simulating flows with rotating geometries.

Highlights

  • Novel Continuous-Discontinuous Galerkin model high-order matrix-free framework

  • Geometric non-conformity at the mortar interface does not introduce additional error

  • Verification study shows optimal convergence rates for high-order elements

  • Validation case shows good experimental agreement for Reynolds numbers from 1 to 2000

  • Energy balance analysis in 3D mixer shows numerical dissipation as low as 1%

Introduction

Rotating geometries are a category of engineering systems that encompass various devices such as spinning disc mixers [1], mixing tanks [2], and wind turbines [3]. Applications of rotating mixing systems comprise, for instance, polymer manufacturing [4], pharmaceutical manufacturing [5], and food processing [6]. Numerical simulation of this type of device is essential for understanding, monitoring and optimizing the aforementioned processes which are often limited by fluid dynamics or other transport processes which are flow-driven (e.g., mass transfer, heat transfer). Simulations are essential since these systems are difficult to investigate experimentally, considering that they may possess complex geometries and exhibit turbulent flows.

From a domain discretization perspective, large mesh deformation can happen due to the rotating motion, and thus remeshing is needed. A feasible alternative to avoid remeshing and maintain accuracy is the Mortar Element Method (MEM), a domain decomposition technique that was initially employed in the context of spectral elements to allow geometric and polynomial non-conformity [7, 8]. It is noteworthy that domain decomposition methods encompass non-overlapping subdomains [9] as well as overlapping meshes for spectral elements [10, 11].

Alternatives to sliding mesh techniques to simulate rotating mixing devices, other approaches comprise (i) rotating frame of reference (RFR) and (ii) the immersed boundary (IB) methods. In the RFR technique, a nonGalilean frame of reference is adopted, and governing equations of fluid motion must account for Coriolis and centrifugal forces [12]. Employing multiple reference frames limits the type of flow that can be simulated, since variables must not significantly change in space or time [13]. In the IB implementation, the rotating part (e.g., impeller) is discretized separately, and its movement is imposed on the immersed boundary by strategies such as the Nitsche method [14, 15] or continuous forcing [2]. A disadvantage of IB methods is that most of them degrade the order of accuracy of the underlying numerical method. Although some works propose high-order compatible IB [16], they often yield linear systems with deteriorated condition number. This deterioration can be highly detrimental at higher Reynolds number, where an accurate representation of the boundary layer flow on the blades of the impeller is necessary to obtain a precise prediction of the transient flow structures.

The MEM consists in subdividing the discretized problem into non-overlapping subdomains, and then weakly enforcing continuity at the generated interface – as an alternative to strong point-wise continuity. This interface between subdomains is discretized into the so-called mortar elements [7]. For rotating mixing devices, two subdomains are generated (rotor and stator), and mortar elements are placed at the rotor-stator interface; the rotor domain contains the rotating element (blades or discs, for instance). Besides employing the MEM for spectral elements [17, 18] [19], other studies extended it to finite element models [9] [20], coupling of spectral and finite elements [21, 22], and of finite and boundary elements [23]. Mortar methods can be classified according to how the boundary integrals at the subdomains intersection are constructed: using Lagrange multipliers or the Nitsche method.

The mortar method with Lagrange multipliers has been frequently used in the context of contact problems [24, 25, 26], since it provides a variationally consistent form of contact imposition even for non-conforming grids [27]. Examples include modeling of point, line and surface contact [27] and large deformation friction [28]. It was also employed to simulate fluid-structure interactions (FSI) such as slender bodies with fluids [29] and to model one-dimensional fibers into three-dimensional fluid flow [30]. Other studies have explored dual-mortar formulations, in which the additional degrees of freedom added by the Lagrange multipliers are condensed [31, 32]. A drawback from employing Lagrange multipliers is the mesh locking that can become an issue, for instance, when significant difference between refinement of subdomains leads to an over-constrained problem [33].

The Nitsche mortar method (or Nitsche mortaring) uses the interior penalty method to enforce continuity at the mortar interface [34]. Gustafsson et al. [35, 36] employed it for elastic contact problems, comparing formulations with different stabilizing techniques. Bansal et al. [37] used the Nitsche method for coupling porous media with Navier-Stokes flow within a finite element discretization, and the Nitsche mortaring approach was also used to model incompressible FSI problems in Burman et al. [14]. In the latter study, the Nitsche penalty and the pressure stabilization terms were identified as the main source of consistency error. Formaggia et al. [38] modeled moving structures in the FSI context by employing an extended finite element method (FEM) formulation coupled with a Nitsche mortar at the contact interface.

In terms of the finite element approximation, both aforementioned categories of MEM implementation usually rely on a Continuous Galerkin (CG) formulation. Discontinuous Galerkin (DG) formulations, nonetheless, have also been widely explored within the context of, for instance, hphp-mortar framework [39] and sliding meshes for computational fluid dynamics (CFD) [40]. Kopera [41] studied mass conservation for Euler equations and non-conforming meshes for both DG and CG cases. The results between DG and CG formulations were almost identical, and it was shown that non-conformity does not introduce additional errors. Examples of DG models for non-conforming meshes were presented for three-dimensional sliding meshes [42], compressible Navier-Stokes spectral element model [40], and high-order approximation discretization of rotating devices [43].

The relative motion between the rotor and stator domains requires that the underlying numerical model accounts for such motion – and hence still satisfies its conservation laws. This is commonly done by employing the Arbitrary Lagrangian Eulerian (ALE) framework [44, 45]. It consists of a hybrid method that combines the advantages of both Lagrangian (simple and accurate definition and monitoring of boundaries) and Eulerian (grid movement independent of fluid particle motion) points of view [46]. This way, the domain velocity is determined arbitrarily, avoiding constant remeshing – and possible errors associated with it. The ALE method is often used in FSI problems such as in Dutta et al. [47], where a dual-mortar FEM model is proposed, and in Zhao et al. [48] where a coupled system of incompressible fluid and porous media flow is presented.

Although the MEM is a variationally consistent and highly accurate tool, it requires increased numerical effort due to the necessity of (i) continuous creation of mortar cells via intersection of the stator and rotor meshes, and (ii) computing additional boundary integrals [49]. In this sense, efficient parallelization is essential to improve the method’s computational cost, especially when simulating large problems. A Message Passing Interface (MPI) parallel implementation for non-conforming grids was discussed by [50], and Mayr & Popp [49] implemented a bins system to improve parallel processing. Another strategy used for high-performance computing (HPC) is the use of matrix-free algorithms [51], which take advantage of computer floating-point operations to reduce memory bandwidth limitations of matrix-based solvers [52]. Recent related works presented high-order matrix-free solvers for the Navier-Stokes in a CG [53] and a DG [54] framework. It is noteworthy that high-order FE approximations are useful techniques to reduce the need for excessive mesh refinement by improving accuracy of coarser grids [55]. Previous works employed high-order methods to simulate sliding domains [56, 43], DG compressible Navier-Stokes equations [40] [54], and DG spectral formulation for hyperbolic-parabolic systems [50]. Leveraging MEM implementation with HPC algorithms such as matrix-free methods is a crucial step to counter-balance the computational cost of mortar operators, and this is yet to be presented in the literature.

Given the need of robust, efficient, and accurate models to simulate rotating mixing devices, this work presents a novel model which combines the mortar element method (MEM) with a matrix-free FEM solver. The model is implemented in Lethe [57], an open-source software based on the deal.II library [58, 59] that solves the incompressible Navier-Stokes equations using an implicit Large-Eddy Simulation (LES) approach and a Continuous Galerkin framework. The matrix-free solver described in [53] is employed, and numerical stability is ensured by employing the Streamline-Upwind/Petrov-Galerkin and Pressure-Stabilizing Petrov-Galerkin methods. Relative rotor-stator motion is accounted for through an ALE formulation, and continuity at the rotor-stator interface is enforced by adding boundary terms in a Discontinuous Galerkin fashion with a Symmetric Interior Penalty Galerkin (SIPG) method [60].

A global coarsening geometric multigrid (GCMG) preconditioner [61] is employed, which is compatible with the matrix-free architecture. To this end, the novelty of our model lies in the combination of the MEM with a matrix-free FEM framework, an approach that overcomes possible memory bandwidth bottlenecks created by using high-order FEM approximations. The computational implementation of the model relies on on-the-fly vector-matrix multiplication in batches, hence taking advantage of modern hardware vectorization and current HPC architectures. This paper is structured as follows. The governing equations and discretized system of the MEM CFD model are presented in § 3, and details on the computational implementation are mentioned in § 4. § 5 contains steady-state and transient verification cases, a three-dimensional scalability study, and a validation example against experimental results for a pitched blade impeller in a baffled stirred tank. Finally, conclusions and further research are discussed.

Governing equations

The incompressible Navier-Stokes in ALE form for a domain Ω\Omega with a boundary ∂Ω\partial\Omega are given by

∇⋅u=0,(0,T]×Ω(1)\nabla\cdot\mathbf{u} = 0, \qquad(0,T] \times\Omega \tag*{(1)}
∂tu+(u⋅∇)u−(uALE⋅∇)u+∇p∗−ν∇2u−f=0,(0,T]×Ω.(2)\partial_t\mathbf{u} + (\mathbf{u} \cdot\nabla)\mathbf{u} - (\mathbf{u}_{\mathrm{ALE}} \cdot\nabla)\mathbf{u} + \nabla p^{*} - \nu\nabla^{2}\mathbf{u} - \mathbf{f} = 0, \qquad(0,T] \times\Omega. \tag*{(2)}

in which u\mathbf{u} is the fluid velocity, uALE\mathbf{u}_{\mathrm{ALE}} is the mesh velocity, p∗=p/ρp^{*} = p/\rho is the kinematic pressure, ρ\rho is the fluid density, ν\nu is the kinematic viscosity, and f\mathbf{f} is a source term.

The space-discretized form of Eq. (1) is obtained using the FEM, where the computational domain Ω‾\overline{\Omega} (triangulation of Ω\Omega) is subdivided into kk elements such that Ω‾=⋃kΩ‾k\overline{\Omega} = \bigcup_k \overline{\Omega}_k and ∂Ω‾\partial\overline{\Omega} is the boundary of Ω‾\overline{\Omega}. By multiplying (1a) and (1b) by test functions qq and v\mathbf{v}, respectively, and summing them, we obtain the following weighted-residual form at the element level:

(q,∇⋅u)Ω‾k+(v,∂tu)Ω‾k+(v,u⋅∇u)Ω‾k−(v,uALE⋅∇u)Ω‾k+(v,∇p∗)Ω‾k−(v,ν∇2u)Ω‾k−(v,f)Ω‾k=0;(3)\begin{aligned} (q,\nabla\cdot\mathbf{u})_{\overline{\Omega}_k} + (\mathbf{v},\partial_t\mathbf{u})_{\overline{\Omega}_k} + (\mathbf{v},\mathbf{u}\cdot\nabla\mathbf{u})_{\overline{\Omega}_k} - (\mathbf{v},\mathbf{u}_{\mathrm{ALE}}\cdot\nabla\mathbf{u})_{\overline{\Omega}_k} \\ {}+(\mathbf{v},\nabla p^{*})_{\overline{\Omega}_k} - (\mathbf{v},\nu\nabla^{2}\mathbf{u})_{\overline{\Omega}_k} - (\mathbf{v},\mathbf{f})_{\overline{\Omega}_k} = 0; \tag*{(3)} \end{aligned}

here we use the following notation for the integrals:

(a,b)Ω‾k=∑k∫ab dΩ‾k.(4)(a,b)_{\overline{\Omega}_k} = \sum_k \int ab\,\mathrm{d}\overline{\Omega}_k. \tag*{(4)}

Mortar element method

The subdivision of rotor and stator regions relies on the Mortar Element Method (MEM) [7, 8] [21], which assumes that the discretized domain Ω‾\overline{\Omega} is subdivided into nn non-overlapping subdomains such that

Ω‾=⋃Ω‾n,Ω‾l∩Ω‾m=∅,∀ l,m, l≠m,n=1,⋯ ,N,(5)\overline{\Omega} = \bigcup\overline{\Omega}^{n}, \qquad\overline{\Omega}^{l} \cap\overline{\Omega}^{m} = \varnothing, \qquad\forall\, l,m,\ l \ne m,\qquad n = 1,\cdots,N, \tag*{(5)}

where NN is the total number of subdomains; for the rotor-stator configuration, N=2N = 2. In this case, the rotor and stator subdomains are indicated by Ωˉ−\bar{\Omega}^{-} and Ωˉ+\bar{\Omega}^{+}, respectively. Figure 1 depicts the cell configuration at the interface between subdomains; we assume that the rotor and stator sides contain the same number of equally spaced elements. Hence, we allow a non-conforming sliding interface in which each stator cell is subdivided by a rotor cell edge at most once. In a dd-dimensional domain, the (d−1)(d - 1)-dimensional mortar cells are created based on the intersections between the edge of the boundary cells and the mortar interface.

Image not extracted

Figure 1. Schematic of curved mortar cells (represented in red) at the interface between rotor (Ωˉ−\bar{\Omega}^{-}) and stator (Ωˉ+\bar{\Omega}^{+}) domains. The mortar interface is partitioned at every intersection with a cell edge, considering both rotor and stator sides. A regular quadrature defined at the boundary cells is mapped into each mortar segment.

Let us indicate the rotor and stator sides at the mortar interface by ∂Ωˉ−\partial\bar{\Omega}^{-} and ∂Ωˉ+\partial\bar{\Omega}^{+}, respectively, such that ∂Ωˉk∗=∂Ωˉk∩(∂Ωˉ−∪∂Ωˉ+)\partial\bar{\Omega}_{k}^{*} = \partial\bar{\Omega}_{k} \cap(\partial\bar{\Omega}^{-} \cup\partial\bar{\Omega}^{+}) refers to element boundaries coincident with the rotor-stator interface and ∂Ωˉk,all∗\partial\bar{\Omega}_{k,\mathrm{all}}^{*} are all the element boundaries of cells at the interface. We define the average {{⋅}}\{\{\cdot\}\} and jump ⟦⋅⟧\llbracket\cdot\rrbracket operators of a generic field ξ\xi as:

{{ξ}}=ξ−+ξ+2,⟦ξ⟧=ξ+−ξ−.(6)\{\{\xi\}\} = \frac{\xi^{-} + \xi^{+}}{2}, \qquad\llbracket\xi\rrbracket= \xi^{+} - \xi^{-}. \tag*{(6)}

The boundary integrals are implemented based on the DG method for incompressible flows proposed by Fehn et al. [62, 63]. Continuity on the velocity and pressure fields is ensured by integrating by parts the pressure and viscous terms in (2); for the former, using a central approximation of the numerical flux, we have:

(v,∇p∗)Ωˉk:=−(∇v,p∗)Ωˉk+(v,{ ⁣{p∗} ⁣}n)∂Ωˉk∗.(7)(\mathbf{v},\nabla p^*)_{\bar{\Omega}_k}:=-(\nabla\mathbf{v},p^*)_{\bar{\Omega}_k}+(\mathbf{v},\{\!\{p^*\}\!\}\mathbf{n})_{\partial\bar{\Omega}_k^*}. \tag*{(7)}

For the viscous term, we employ the symmetric interior penalty Galerkin (SIPG) method [64, 65] to obtain the following weak form:

−(v,ν∇2u)Ωˉk:=(∇v,ν∇u)Ωˉk−(∇v,ν2[ ⁣[u] ⁣]⊗n)∂Ωˉk∗−(v,ν{ ⁣{∇u} ⁣}⋅n)∂Ωˉk∗+(v,ντk[ ⁣[u] ⁣])∂Ωˉk∗,(8)-(\mathbf{v},\nu\nabla^2\mathbf{u})_{\bar{\Omega}_k}:=(\nabla\mathbf{v},\nu\nabla\mathbf{u})_{\bar{\Omega}_k}-(\nabla\mathbf{v},\frac{\nu}{2}[\![\mathbf{u}]\!]\otimes\mathbf{n})_{\partial\bar{\Omega}_k^*}-(\mathbf{v},\nu\{\!\{\nabla\mathbf{u}\}\!\}\cdot\mathbf{n})_{\partial\bar{\Omega}_k^*}+(\mathbf{v},\nu\tau_k[\![\mathbf{u}]\!])_{\partial\bar{\Omega}_k^*}, \tag*{(8)}

where the penalty parameter τk\tau_k for each element is defined according to the work of Hillewaert [66]:

τk=F(P+1)2A(∂Ωˉk,all∗\∂Ωˉk∗)/2+A(∂Ωˉk∗)V(Ωˉk).(9)\tau_k=F(\mathcal{P}+1)^2\frac{A(\partial\bar{\Omega}_{k,\mathrm{all}}^*\backslash\partial\bar{\Omega}_k^*)/2+A(\partial\bar{\Omega}_k^*)}{V(\bar{\Omega}_k)}. \tag*{(9)}

In this expression, FF is a factor that controls the amount of penalization (generally chosen in the range of 1 to 10 [60]), P\mathcal{P} is the polynomial degree of the FE approximation, V(Ωˉk)=∫dΩˉkV(\bar{\Omega}_k)=\int\mathrm{d}\bar{\Omega}_k is the element volume, and AA is the surface area of the interior (∂Ωˉk,all∗\∂Ωˉk∗)(\partial\bar{\Omega}_{k,\mathrm{all}}^*\backslash\partial\bar{\Omega}_k^*) and exterior (∂Ωˉk∗)(\partial\bar{\Omega}_k^*) element boundaries.

Regarding the mass conservation term, we integrate it by parts twice and adopt the strong DG formulation to improve mass conservation given by [62]:

(q,∇⋅u)Ωˉk:=(q,∇⋅u)Ωˉk−(q,12[ ⁣[u] ⁣]⋅n)∂Ωˉk∗.(10)(q,\nabla\cdot\mathbf{u})_{\bar{\Omega}_k}:=(q,\nabla\cdot\mathbf{u})_{\bar{\Omega}_k}-(q,\frac{1}{2}[\![\mathbf{u}]\!]\cdot\mathbf{n})_{\partial\bar{\Omega}_k^*}. \tag*{(10)}

Substituting relations (6), (7), and (9) into Eq. (3), we have:

FNS∗:=(q,∇⋅u)Ωˉ−(q,12[ ⁣[u] ⁣]⋅n)∂Ωˉ∗+(v,∂tu)Ωˉ+(v,u⋅∇u)Ωˉ−(v,uALE⋅∇u)Ωˉ−(∇v,p∗)Ωˉ+(v,{ ⁣{p∗} ⁣}n)∂Ωˉ∗+(∇v,ν∇u)Ωˉ−(∇v,ν2[ ⁣[u] ⁣]⊗n)∂Ωˉ∗−(v,ν{ ⁣{∇u} ⁣}⋅n)∂Ωˉ∗+(v,ντk[ ⁣[u] ⁣])∂Ωˉ∗−(v,f)Ωˉ.(11)\begin{aligned} F_{NS}^*&:=(q,\nabla\cdot\mathbf{u})_{\bar{\Omega}}-(q,\frac{1}{2}[\![\mathbf{u}]\!]\cdot\mathbf{n})_{\partial\bar{\Omega}^*}+(\mathbf{v},\partial_t\mathbf{u})_{\bar{\Omega}}+(\mathbf{v},\mathbf{u}\cdot\nabla\mathbf{u})_{\bar{\Omega}}\\ &\quad-(\mathbf{v},\mathbf{u}_{\mathrm{ALE}}\cdot\nabla\mathbf{u})_{\bar{\Omega}}-(\nabla\mathbf{v},p^*)_{\bar{\Omega}}+(\mathbf{v},\{\!\{p^*\}\!\}\mathbf{n})_{\partial\bar{\Omega}^*}+(\nabla\mathbf{v},\nu\nabla\mathbf{u})_{\bar{\Omega}}\\ &\quad-(\nabla\mathbf{v},\frac{\nu}{2}[\![\mathbf{u}]\!]\otimes\mathbf{n})_{\partial\bar{\Omega}^*}-(\mathbf{v},\nu\{\!\{\nabla\mathbf{u}\}\!\}\cdot\mathbf{n})_{\partial\bar{\Omega}^*}+(\mathbf{v},\nu\tau_k[\![\mathbf{u}]\!])_{\partial\bar{\Omega}^*}-(\mathbf{v},\mathbf{f})_{\bar{\Omega}}. \tag*{(11)} \end{aligned}

Numerical stabilization

We ensure the stabilization of the weak form by using the Streamline-Upwind/Petrov-Galerkin (SUPG) and Pressure-Stabilizing Petrov-Galerkin (PSPG) approaches [57], and the terms aSUPGa_{\mathrm{SUPG}}, aPSPGa_{\mathrm{PSPG}} are added into the weak form:

F^NS∗:=FNS∗+(τu(u−uALE)⋅∇v,Ru)Ω‾k+(τu∇q,Ru)Ω‾k.(12)\widehat{F}^{*}_{NS} := F^{*}_{NS} + \left(\tau_{\mathrm{u}}(\mathbf{u}-\mathbf{u}_{\mathrm{ALE}})\cdot\nabla\mathbf{v},\mathcal{R}_{\mathrm{u}}\right)_{\overline{\Omega}_{k}} + \left(\tau_{\mathrm{u}}\nabla q,\mathcal{R}_{\mathrm{u}}\right)_{\overline{\Omega}_{k}} . \tag*{(12)}

The strong residual of the momentum conservation Ru\mathcal{R}_{\mathrm{u}} is defined as:

Ru=∂tu+(u⋅∇)u−(uALE⋅∇)u+∇p∗−ν∇2u−f,(13)\mathcal{R}_{\mathrm{u}} = \partial_{\mathrm{t}}\mathbf{u} + (\mathbf{u}\cdot\nabla)\mathbf{u} - (\mathbf{u}_{\mathrm{ALE}}\cdot\nabla)\mathbf{u} + \nabla p^{*} - \nu\nabla^{2}\mathbf{u} - \mathbf{f}, \tag*{(13)}

and τu\tau_{\mathrm{u}} is a stabilization parameter defined according to Tezduyar [67]:

τu=[(1Δt)2+(2∥u−uALE∥hconv)2+9(4νhdiff2)2]−1/2(14)\tau_{\mathrm{u}} = \left[\left(\frac{1}{\Delta t}\right)^{2} + \left(\frac{2\lVert\mathbf{u}-\mathbf{u}_{\mathrm{ALE}}\rVert}{h_{\mathrm{conv}}}\right)^{2} + 9\left(\frac{4\nu}{h_{\mathrm{diff}}^{2}}\right)^{2}\right]^{-1/2} \tag*{(14)}

where Δt\Delta t is the time step and hconv=hdiff=hh_{\mathrm{conv}}=h_{\mathrm{diff}}=h is the element size given by the diameter of the corresponding area-equivalent circle.

Finite element discretization

Considering the Galerkin method, the weak form in Eq. (11) can be discretized in space by employing the following representation of v\mathbf{v}, u\mathbf{u}, qq, p∗p^{*}:

v=∑i=1Nuviϕi,u=∑j=1Nuujϕj,q=∑k=1Npqkψk,p∗=∑l=1Nppl∗ψl,(15)\mathbf{v} = \sum_{i=1}^{N_{\mathrm{u}}} v_i\boldsymbol{\phi}_i,\qquad\mathbf{u} = \sum_{j=1}^{N_{\mathrm{u}}} u_j\boldsymbol{\phi}_j,\qquad q = \sum_{k=1}^{N_{\mathrm{p}}} q_k\psi_k,\qquad p^{*} = \sum_{l=1}^{N_{\mathrm{p}}} p_l^{*}\psi_l, \tag*{(15)}

in which ϕi\boldsymbol{\phi}_i, ϕj\boldsymbol{\phi}_j, ψk\psi_k, ψl\psi_l are Lagrange polynomials of degree P\mathcal{P}, and viv_i, uju_j, qkq_k, pl∗p_l^{*} are nodal values. The nonlinear system is solved using Newton’s method, where a linear system of equations of the form:

J[δuδp∗]=−R(16)\mathcal{J}\begin{bmatrix}\delta\mathbf{u}\\\delta p^{*}\end{bmatrix} = -\mathcal{R} \tag*{(16)}

is solved. The weak form of the Jacobian and residual are given by J\mathcal{J} and R\mathcal{R}, respectively, and [δu,δp∗]T[\delta\mathbf{u},\delta p^{*}]^{\mathsf{T}} are the computed increments. Writing the linear system in matrix form, we have that:

\begin{bmatrix} \widehat{\mathbf{A}} & \widehat{\mathbf{B}}\\ \widehat{\mathbf{C}} & \widehat{\mathbf{D}} \end{bmatrix} \begin{bmatrix} \delta\mathbf{u}\\ \delta p^{*} \end{bmatrix} =- \begin{bmatrix} \mathbf{R}_{\mathrm{u}}\\ R_{\mathrm{p}} \end{bmatrix}. \tag*{(17)}\end{bmatrix}

The stabilized Jacobian sub-matrices are given by:

A^ijk=Aijk+(τu(u−uALE)⋅∇ϕi,∂tϕj+ϕj⋅∇u+(u−uALE)⋅∇ϕj−ν∇2ϕj)Ωˉk+(τu(ϕj−uALE)⋅∇ϕi,∂tu+(u−uALE)⋅∇u−ν∇2u+∇p∗−f)Ωˉk(18)\hat{A}_{ij}^{k}=A_{ij}^{k} +\left(\tau_u(\mathbf{u}-\mathbf{u}_{\mathrm{ALE}})\cdot\nabla\phi_i,\partial_t\phi_j+\phi_j\cdot\nabla\mathbf{u}+(\mathbf{u}-\mathbf{u}_{\mathrm{ALE}})\cdot\nabla\phi_j-\nu\nabla^2\phi_j\right)_{\bar{\Omega}_k} +\left(\tau_u(\phi_j-\mathbf{u}_{\mathrm{ALE}})\cdot\nabla\phi_i,\partial_t\mathbf{u}+(\mathbf{u}-\mathbf{u}_{\mathrm{ALE}})\cdot\nabla\mathbf{u}-\nu\nabla^2\mathbf{u}+\nabla p^*-\mathbf{f}\right)_{\bar{\Omega}_k} \tag*{(18)}
B^ijk=Bijk+(τu(u−uALE),∇ψj)Ωˉk,(19)\hat{B}_{ij}^{k}=B_{ij}^{k}+\left(\tau_u(\mathbf{u}-\mathbf{u}_{\mathrm{ALE}}),\nabla\psi_j\right)_{\bar{\Omega}_k}, \tag*{(19)}
C^ijk=Cijk+(τu∇ψi,∂tϕj+ϕj⋅∇u+(u−uALE)⋅∇ϕj−ν∇2ϕj)Ωˉk,(20)\hat{C}_{ij}^{k}=C_{ij}^{k}+\left(\tau_u\nabla\psi_i,\partial_t\phi_j+\phi_j\cdot\nabla\mathbf{u}+(\mathbf{u}-\mathbf{u}_{\mathrm{ALE}})\cdot\nabla\phi_j-\nu\nabla^2\phi_j\right)_{\bar{\Omega}_k}, \tag*{(20)}
D^ijk=Dijk+(τu∇ψi,∇ψj)Ωˉk,(21)\hat{D}_{ij}^{k}=D_{ij}^{k}+\left(\tau_u\nabla\psi_i,\nabla\psi_j\right)_{\bar{\Omega}_k}, \tag*{(21)}

where

Aijk=(ϕi,∂tϕj)Ωˉk+(ϕi,∇u⋅ϕj)Ωˉk+(ϕi,u⋅∇ϕj)Ωˉk−(ϕi,uALE⋅∇ϕj)Ωˉk+(∇ϕi,ν∇ϕj)Ωˉk−(∇ϕi,ν2⟦ϕj⟧⊗n)∂Ωˉk∗−(ϕi,ν{ ⁣{∇ϕj} ⁣}⋅n)∂Ωˉk∗+(ϕi,ντk⟦ϕj⟧)∂Ωˉk∗,(22)A_{ij}^{k}=(\phi_i,\partial_t\phi_j)_{\bar{\Omega}_k}+(\phi_i,\nabla\mathbf{u}\cdot\phi_j)_{\bar{\Omega}_k}+(\phi_i,\mathbf{u}\cdot\nabla\phi_j)_{\bar{\Omega}_k}-(\phi_i,\mathbf{u}_{\mathrm{ALE}}\cdot\nabla\phi_j)_{\bar{\Omega}_k} +(\nabla\phi_i,\nu\nabla\phi_j)_{\bar{\Omega}_k} -\left(\nabla\phi_i,\frac{\nu}{2}\llbracket\phi_j\rrbracket\otimes\mathbf{n}\right)_{\partial\bar{\Omega}_k^*} -\left(\phi_i,\nu\{\!\{\nabla\phi_j\}\!\}\cdot\mathbf{n}\right)_{\partial\bar{\Omega}_k^*} +\left(\phi_i,\nu\tau_k\llbracket\phi_j\rrbracket\right)_{\partial\bar{\Omega}_k^*}, \tag*{(22)}
Bijk=−(∇ϕi,ψj)Ωˉk+(ϕi,{ ⁣{ψj} ⁣}n)∂Ωˉk∗,(23)B_{ij}^{k}=-(\nabla\phi_i,\psi_j)_{\bar{\Omega}_k}+\left(\phi_i,\{\!\{\psi_j\}\!\}\mathbf{n}\right)_{\partial\bar{\Omega}_k^*}, \tag*{(23)}
Cijk=(ψi,∇⋅ϕj)Ωˉk−(ψi,12⟦ϕj⟧⋅n)∂Ωˉk∗,(24)C_{ij}^{k}=(\psi_i,\nabla\cdot\phi_j)_{\bar{\Omega}_k}-\left(\psi_i,\frac{1}{2}\llbracket\phi_j\rrbracket\cdot\mathbf{n}\right)_{\partial\bar{\Omega}_k^*}, \tag*{(24)}
Dijk=0,(25)D_{ij}^{k}=0, \tag*{(25)}

and Ru\mathbf{R}_u, Rp\mathbf{R}_p are the discretized residual terms.

Software implementation

The mortar method is implemented in Lethe [57], an open-source CFD software that is based on the deal.II finite element library [59] and uses p4est for mesh management [68], Trilinos for numerical linear algebra [69], and MPI for parallelization. Although the present work relies on the matrix-free implementation, the MEM implementation is compatible with both matrix-based and matrix-free architectures; more on the implementation and verification of the matrix-free solver can be found in [53].

Three main components are used to integrate the mortar method: Mortar Manager, Coupling Operator, and Coupling Evaluation. The Mortar Manager is responsible for storing information regarding mesh alignment, mortar-cell indices, weights and normal vectors of the quadrature points. The Mortar Manager accounts for the generation of curved mortar cells within the circular geometry of the rotor-stator interface. Such interface is approximated with high-order polynomials and hence the normals are determined exactly. The Coupling Operator stores information corresponding to each interface side (∂Ω‾+\partial\overline{\Omega}^{+} and ∂Ω‾−\partial\overline{\Omega}^{-}, as described in § 3.1), computes the cell penalty factor, and updates the constraints related to the complete system of equations. Lastly, the Coupling Evaluation computes the mortar terms represented in Eq. (11).

Grid generation and updated configuration

The grids for the rotor and stator subdomains are generated separately and merged into a unique triangulation. We assume that the number of elements on both sides of the mortar interface is equal, and that such elements are equally sized and spaced. In three-dimensional cases, this restriction is enforced in the planes perpendicular and parallel to the rotation axis. As mentioned in § 3, the displacement of the rotor domain mesh (due to an imposed angular velocity) may lead to non-conforming configurations of the rotor-stator interface, and thus, each boundary cell may be connected to up to two mortar cells. Although this assumption of equal angular length of the boundary cells at the mortar interface seems restrictive, it has been a common choice in MEM applications [42, 56, 40] and could be alleviated in future work by allowing more than two mortar cells to be connected to a single regular cell in either Ω‾+\overline{\Omega}^{+} or Ω‾−\overline{\Omega}^{-}.

The rotated rotor domain is obtained by a mapping of the initial configuration, using the following relation for the transformation of each node position:

x=Rx0(26)\mathbf{x} = \mathbf{R}\mathbf{x}_0 \tag*{(26)}

where x0\mathbf{x}_0 is the initial node coordinate, x\mathbf{x} is the updated coordinate, and R\mathbf{R} is the rotation matrix with respect to the origin. The latter is defined for two-dimensional cases as:

R:=[cos⁡[θ(t)]−sin⁡[θ(t)]sin⁡[θ(t)]cos⁡[θ(t)]](27)\mathbf{R} := \begin{bmatrix} \cos[\theta(t)] & -\sin[\theta(t)] \\ \sin[\theta(t)] & \cos[\theta(t)] \end{bmatrix} \tag*{(27)}

in three dimensions, R\mathbf{R} is given by:

R:=cos⁡[θ(t)] I+sin⁡[θ(t)] W+(1−cos⁡[θ(t)]) r⊗r(28)\mathbf{R} := \cos[\theta(t)]\,\mathbf{I} + \sin[\theta(t)]\,\mathbf{W} + (1-\cos[\theta(t)])\,\mathbf{r}\otimes\mathbf{r} \tag*{(28)}

where r\mathbf{r} is the rotation axis vector around the origin, I\mathbf{I} is the identity tensor, and W\mathbf{W} is the skew symmetric tensor of r\mathbf{r}.

Creation of mortar operators

Let us consider the following boundary integral computed at mortar cells, as indicated in Eq. (22):

(ϕi,ντk⟦ϕj⟧)∂Ωk∗≈∑q=1nq(ϕi(q) ντk⟦ϕj(q)⟧) wq(det⁡Jk),(29)(\phi_i,\nu\tau_k\llbracket\phi_j\rrbracket)_{\partial\Omega_k^*}\approx\sum_{q=1}^{n_q}(\phi_i(q)\,\nu\tau_k\llbracket\phi_j(q)\rrbracket)\,w_q(\det\mathbf{J}_k), \tag*{(29)}

where we approximate the integral with a summation over Gauss quadrature points qq (q=1,…,nqq=1,\ldots,n_q); wqw_q is the weight of each quadrature point, and Jk\mathbf{J}_k is the Jacobian matrix of the transformation from the Cartesian coordinates x\mathbf{x} to the mapped reference system ξ\boldsymbol{\xi}.

As indicated in Figure 1, the quadrature and weights at the mortar cells are obtained by mapping a reference quadrature into each mortar segment. The determinant of the Jacobian of this transformation is computed as:

det⁡Jk=Rδ0δ1,(30)\det\mathbf{J}_k=R\delta_0\delta_1, \tag*{(30)}

in which RR is the radius of the in-plane rotor-stator interface, δ0\delta_0 is the angle corresponding to the mortar segment at the plane perpendicular to the rotation axis, given by:

δ0={2πRnR,for aligned meshes,α or 2πRnR−α,for misaligned meshes,(31)\delta_0= \begin{cases} \frac{2\pi R}{n_R}, & \text{for aligned meshes},\\ \alpha\ \text{or}\ \frac{2\pi R}{n_R}-\alpha, & \text{for misaligned meshes}, \end{cases} \tag*{(31)}

where α\alpha is the angle between two vertices in the rotor and stator sides (see misalignment example in Figure 1), and nRn_R is the in-plane number of subdivisions. In (29), δ1\delta_1 is the cell height along the rotation axis, given by:

δ1=HnH,(32)\delta_1=\frac{H}{n_H}, \tag*{(32)}

where HH is the out-of-plane dimension and nHn_H is the corresponding number of subdivisions. Since we assume that the element size is the same in the in-plane and out-of-plane configurations, the Jacobian of the transformation is constant.

The integration is performed in two steps. First, we evaluate the variables at the mortar quadrature points and store the result. For instance, to compute the jump values ⟦ϕj(q)⟧\llbracket\phi_j(q)\rrbracket we evaluate the ϕj(q)\phi_j(q) at both sides of the interface. For the boundary terms containing the normal unit vector n\mathbf{n}, it is decomposed into n−\mathbf{n}^{-} and n+\mathbf{n}^{+}. Communication is the next step, where data is exported from local (Ω‾−)(\overline{\Omega}^{-}) to ghost (Ω‾+)(\overline{\Omega}^{+}) side. After that, we load the stored data and perform the integration of the boundary term. The matrix-free evaluation is performed via the algorithm described in [70, 71], which aims to optimize repeated computations on cached data to adapt to the capabilities of modern hardware. The computation of the remaining boundary integrals at the mortar interface follow the same steps as the one herein described.

Mesh parallel decomposition

The mesh parallel decomposition relies on the p4est library [68]. It partitions space-fitting curves, and allows the attribution of different cell weights to be taken into account during load balancing. In the current application, the space-fitting curve is the composition of the stator and rotor sides of the mortar interface. Figure 2 depicts an example of domain partitioning over processes during mesh refinement of a steady case, while Figure 3 shows the partitioning as the rotor domain mapping is rotated. We do not attempt to keep the rotor-stator interface cells within a single process. Child cells do not necessarily remain on the same process as their original unrefined cell, while rotated cells are still owned by the same initial rank.

Image not extracted

Figure 2. Cell ownership as the mesh is refined; the colors indicate the seven subdomains (ranks), and an iso-parametric mapping is used.

Image not extracted

Figure 3. Cell ownership as the rotor mesh is rotated anti-clockwise; the colors indicate the seven subdomains (ranks), and an iso-parametric mapping is used.

Geometric multigrid preconditioner

The GCMG preconditioner builds a hierarchy of discretizations through geometric coarsening of the mesh (hh) and reduction of the polynomial degree (P\mathcal{P}), applying a smoother on each level and a coarse-grid solver on the coarsest one. Available coarse-grid solvers in Lethe include a direct solver, an algebraic multigrid (AMG) solver, an incomplete LU factorization (ILU), and an iterative generalized minimal residual method (GMRES), preconditioned by AMG or ILU. As smoother, we use an inverse diagonal (ID) or the Additive Schwarz method (ASM) accelerated by a relaxation scheme [53]. The GCMG preconditioner does not require global matrices to be assembled, and hence it is compatible with the matrix-free architecture. More details on the GCMG deal.II architecture can be found in [61], while its implementation in Lethe is further described in [53].

The contributions from the mortar interface integrals need to be accounted on each level, in particular, in the smoother and coarse-grid solver. To this end, the relevant element matrices (or their diagonals) are reconstructed column-by-column, and hence the storage of full matrices is avoided – a common approach in matrix-free algorithms. The corresponding pseudocodes are presented in Algorithms 1 and 2.

Numerical examples

In this section we present study cases to verify and validate the MEM implementation within Lethe. A steady-state two-dimensional example is described in § 5.1, where a manufactured solution is employed so that the convergence rate of high-order polynomial approximations can be observed. In § 5.2, a transient two-dimensional Taylor-Green vortex example is used to verify the convergence order of viscid and inviscid flows. In both two-dimensional cases, we study the change in error norms due to different Algorithm 1 Reconstruction of mortar contribution to matrix diagonal. The loop over cells cmc_m refers to the cells located at either side of the mortar interface.

  • for each cell cmc_m do

    • for each local DoF ii do

      • Create basis vector eie_i associated with the shape function NiN_i

      • Evaluate eie_i at quadrature points

      • Integrate mortar contributions

      • Acm(i,i)←iA_{c_m}(i,i) \leftarrow i-th entry of resulting vector

    • end for

    • Assemble global diagonal

  • end for

Algorithm 2 Reconstruction of mortar contribution to matrix AA. The loop over cells cmc_m refers to the cells located at either side of the mortar interface.

  • for each cell cmc_m do

    • for each local DoF ii do

      • Create basis vector eie_i associated with the shape function NiN_i

      • Evaluate eie_i at quadrature points

    • end for

  • end for

  • Communicate evaluated values to ghost cells

  • for each cell cmc_m do

    • for each relevant DoF ii do

      • Integrate mortar contributions

      • for each local DoF jj do

        • Acm(j,i)←jA_{c_m}(j,i) \leftarrow j-th entry of resulting vector

      • end for

    • end for

    • Assemble global matrix

  • end for

mesh alignment patterns at the interface. The performance of the solver is assessed in § 5.3, where a three-dimensional example is used to show strong and weak scalability of the model. Following that, § 5.4 and § 6 showcases simulations of stirred mixing tanks in two- and three-dimensions, respectively, to investigate mesh convergence and validate the model with existing numerical and experimental data. In the following examples, unless otherwise stated, we solve the linearized Navier-Stokes equations using the GMRES method, a GCMG preconditioner [72, 61] and an inverse diagonal smoother as a relaxation scheme. In the transient examples, a second-order implicit backward-difference time integration scheme (BDF2) is employed. The following examples employ equal high-order elements QPQPQ_{\mathcal{P}}Q_{\mathcal{P}} (P≥2\mathcal{P} \ge2, where P\mathcal{P} is the polynomial order) for the velocity and pressure fields. While the SUPG-PSPG is a robust stabilization for the CG framework of the Navier-Stokes solver, the introduction of the DG mortar interface generated pressure instability for the linear polynomial approximation Q1Q1Q_1Q_1. Additional tests have shown that adding a bubble-type enrichment function [73] to the velocity field recovers the interface continuity in the matrix-based solver. Numerical examples considering enriched Q1Q1Q_1Q_1 elements, as well as a discussion on alternative stabilization methods for the linear approximation, will be addressed in future work.

Steady-state manufactured solution

We verify the MEM implementation by simulating a two-dimensional steady-state problem with the Method of Manufactured Solutions (MMS). The manufactured problem is:

u=−2sin⁡(πx)2cos⁡(πy)sin⁡(πy),(33)u = -2\sin(\pi x)^2\cos(\pi y)\sin(\pi y), \tag*{(33)}
v=2cos⁡(πx)sin⁡(πx)sin⁡(πy)2,(34)v = 2\cos(\pi x)\sin(\pi x)\sin(\pi y)^2, \tag*{(34)}
p∗=sin⁡(πx)sin⁡(πy),(35)p^* = \sin(\pi x)\sin(\pi y), \tag*{(35)}

where uu, vv are the xx and yy components of the velocity vector u\mathbf{u}. The problem domain is a combination of a plate with a hole and a circle, which yields a simple merged geometry that still allows us to evaluate the MEM accuracy (see Figure 4a). The combined rotor-stator domain is defined in [−1,1]×[−1,1][-1,1] \times[-1,1], with the circular interface located at a radial distance of 0.5 from the center point. In this example, no rotation velocity is imposed on the rotor domain (ω=0\omega= 0).

The simulation is carried out for a kinematic viscosity of ν=1\nu= 1, four refinement levels (the refinement level l=1l = 1 contains 80 elements), and polynomial approximations from P=2\mathcal{P} = 2 to P=5\mathcal{P} = 5. In this steady case, the time-dependent term indicated in Eq. (1) is neglected. Figure 5 depicts the L2L_2 error norms, where we notice the optimal O(hP+1)O(h^{\mathcal{P}+1}), O(hP)O(h^\mathcal{P}) rates for velocity and pressure fields, respectively. Since the rotor-stator mesh is non-uniform, the reported values of h~\tilde{h} refer to a characteristic mesh size based on the average element area. The L2L_2 norms of the error are compared to reference values (indicated as (R\mathrm{R}) in Figure 5), in which the same parameters and mesh are used to simulate the MMS case without a mortar interface. Said error norms are identical for the subdivided rotor-stator domain and for the reference case.

Image not extracted

Figure 5. L2L_2 norm of the error of velocity (left) and pressure (right) fields for the two-dimensional MMS case. In the reference case, the mesh in Figure 4a is used without a mortar interface. For the rotor-stator mesh, no rotor rotation was imposed.

Image not extracted

(a) Mesh configuration (refinement level l=4l = 4) used in the two-dimensional MMS simulation (§ 5.1) and TGV case (§ 5.2). The corresponding domain is Ω=[−1,1]×[−1,1]\Omega= [-1,1] \times[-1,1] and the rotor-stator interface is placed at a radial distance of 0.5 from the center point.

Image not extracted

(b) Mesh configuration (refinement level l=2l = 2) used in three-dimensional MMS scalability study (§ 5.3). The corresponding domain is Ω=[−1,1]×[−1,1]×[0,2]\Omega= [-1,1] \times[-1,1] \times[0,2] and the cylindrical mortar interface is located at radial distance of 0.5 from the rotation axis zz.

Figure 4. Mesh discretization for (a) two-dimensional and (b) three-dimensional manufactured solution examples.

Considering the same problem, we analyze the dependence of the error on the rotor mesh rotation. The goal is to verify whether the alignment of the mortar cells with respect to the stator boundary cells (see illustration in Figure 1) interferes in the accuracy of the solution. Considering that the angular length of each interface cell in the coarsest mesh is θmax⁡=0.3926\theta_{\max} = 0.3926 rad, we take 200 sampling values in the interval [0,θmax⁡][0,\theta_{\max}] to verify the corresponding error norms. Figure 6 depicts the L2L_2 error norm for four refinement levels, considering Q2Q2Q_2Q_2 and Q3Q3Q_3Q_3 elements. Regardless of the refinement level, the error norms for both velocity and pressure fields remain constant, confirming that the geometric incompatibility generated at the interface due to mortar cell misalignment does not lead to loss of convergence properties. This conclusion is in alignment with the fact that a reference quadrature is mapped into each mortar segment intersected by the boundary cells edges (Figure 1), and thus the quadrature order is preserved regardless of the mortar element size. The same behavior has been observed in [42] for a DG formulation.

Image not extracted

Figure 6. L2L_2 error norm for the two-dimensional MMS problem considering different initial rotation angles of the rotor mesh. The points in each curve are taken from the interval [0,θmax⁡][0,\theta_{\max}], where θmax⁡=π/8\theta_{\max} = \pi/8 is the cell angular length at the mortar interface for the coarsest refinement level.

Taylor-Green vortex

The transient MEM implementation is verified with a two-dimensional Taylor-Green vortex (TGV) case. The same domain geometry and mesh discretization from § 5.1 is employed, as indicated in Figure 4a, and an angular velocity ω\omega is prescribed in the rotor domain. The initial condition is given by:

u=cos⁡(x)sin⁡(y),(36)u = \cos(x)\sin(y), \tag*{(36)}
v=−sin⁡(x)cos⁡(y),(37)v = -\sin(x)\cos(y), \tag*{(37)}
p=−14[cos⁡(2x)+cos⁡(2y)],(38)p = -\frac{1}{4}\left[\cos(2x) + \cos(2y)\right], \tag*{(38)}

in which uu, vv are the xx and yy components of the velocity vector u\mathbf{u}, respectively. The simulation considers a kinematic viscosity of ν=1\nu= 1 and a time step of Δt=10−4 s\Delta t = 10^{-4}\,\mathrm{s} during tend=0.1 st_{end} = 0.1\ \mathrm{s}. The corresponding analytical solution is given by:

u=exp⁡(−2νt)cos⁡(x)sin⁡(y),(39)u = \exp(-2\nu t)\cos(x)\sin(y), \tag*{(39)}
v=−exp⁡(−2νt)sin⁡(x)cos⁡(y).(40)v = -\exp(-2\nu t)\sin(x)\cos(y). \tag*{(40)}

Figure 7 shows velocity L2L_2 error norms for a case with no rotor mesh rotation (ω=0\omega= 0), as well as other randomly chosen values of rotor angular velocity. The expected convergence rate O(hP+1)\mathcal{O}(h^{P+1}) is obtained for all ω\omega in the Q3Q3Q_3Q_3 case. For P=2P = 2, convergence is deteriorated due to the odd-degree structure of the velocity Taylor expansion. Additional tests showed that the same pattern is observed for fourth-order polynomial shape functions and that fifth-order polynomials (Q5Q5Q_5Q_5) recovered the expected order O(hP+1)\mathcal{O}(h^{P+1}). Convergence rates for ω=0.001\omega= 0.001 to ω=0.3 rad/s\omega= 0.3\ \mathrm{rad/s} are almost identical to the case with no mesh rotation, which can be noticed as the error markers overlap. The difference in error norms spans from zero to 10−410^{-4}, decreasing with element size.

Image not extracted

Figure 7. L2L_2 error of velocity field in the two-dimensional TGV problem for different angular velocities of the rotor mesh, considering viscid (left, ν=1\nu= 1) and inviscid (right, ν=0\nu= 0) cases. The shaded gray triangles indicate the convergence rate for all angular velocity values, which have units of rad/s.

To verify the accuracy of the scheme in the absence of viscous dissipation effects, we repeated the convergence analysis for an inviscid case (Re→∞Re \to\infty), assuming ν=0\nu= 0. Figure 7 depicts the convergence rates for the velocity field considering the same rotating speeds as in the viscid case. The difference in error norms also stays in the order of 10−410^{-4} when we compare ω≠0\omega\ne0 to the stationary case, and we observe the same convergence trends for P=2,3P = 2, 3 from the viscid case.

Considering a diagonal cross-section from (−1,−1)(-1,-1) to (1,1)(1,1), we compare the numerical results with the analytical solution as depicted in Figure 8, and good agreement is achieved. In this case, the finest refinement level reported in the convergence study (Figure 7), i.e. 5120 elements, is employed. The computed absolute errors vary between 10−410^{-4} and 10−1310^{-13}, and higher values are reported in the stator mesh, where the average element size is larger than in the rotor mesh.

Image not extracted

Figure 8. Comparison between analytical and numerical solution for two-dimensional TGV case in a diagonal cross-section from (x=−1,y=−1)(x=-1,y=-1) to (x=1,y=1)(x=1,y=1).

Performance and scalability tests

To verify scalability of the matrix-free MEM implementation, we study a three-dimensional MMS case in which the mortar interface is introduced by combining a cylinder and an extruded plate with hole, as depicted in

Figure 4b. The manufactured solution is expressed as:

u=2π2[−3cos⁡(2πx)+2]sin⁡(πy)sin⁡(πz)cos⁡(πy)cos⁡(πz)+π[2cos⁡(πy)2−cos⁡(πz)2]sin⁡(πx)3sin⁡(πy)2sin⁡(πz)2cos⁡(πx),(41)\begin{aligned} u &= 2\pi^2[-3\cos(2\pi x)+2]\sin(\pi y)\sin(\pi z)\cos(\pi y)\cos(\pi z) \\ &\quad+\pi[2\cos(\pi y)^2-\cos(\pi z)^2]\sin(\pi x)^3\sin(\pi y)^2\sin(\pi z)^2\cos(\pi x), \tag*{(41)} \end{aligned}
v=2π2[−3cos⁡(2πy)+2]sin⁡(πx)sin⁡(πz)cos⁡(πx)cos⁡(πz)+π[2cos⁡(πx)2−cos⁡(πz)2]sin⁡(πx)2sin⁡(πy)3sin⁡(πz)2cos⁡(πy),(42)\begin{aligned} v &= 2\pi^2[-3\cos(2\pi y)+2]\sin(\pi x)\sin(\pi z)\cos(\pi x)\cos(\pi z) \\ &\quad+\pi[2\cos(\pi x)^2-\cos(\pi z)^2]\sin(\pi x)^2\sin(\pi y)^3\sin(\pi z)^2\cos(\pi y), \tag*{(42)} \end{aligned}
w=4π2[3cos⁡(2πz)−2]sin⁡(πx)sin⁡(πy)cos⁡(πx)cos⁡(πy)+2π[cos⁡(πx)2+cos⁡(πy)2]sin⁡(πx)2sin⁡(πy)2sin⁡(πz)3cos⁡(πz),(43)\begin{aligned} w &= 4\pi^2[3\cos(2\pi z)-2]\sin(\pi x)\sin(\pi y)\cos(\pi x)\cos(\pi y) \\ &\quad+2\pi[\cos(\pi x)^2+\cos(\pi y)^2]\sin(\pi x)^2\sin(\pi y)^2\sin(\pi z)^3\cos(\pi z), \tag*{(43)} \end{aligned}
p=0.(44)p=0. \tag*{(44)}

The steady-state problem is tested with two mesh configurations with the number of cells at the coarse level equal to l0=18l_0=18 and l0=40l_0=40, for P=2\mathcal{P}=2 and P=3\mathcal{P}=3. The corresponding number of cells and degrees of freedom (DoFs) are reported in Table 1. For the Q3Q3Q_3Q_3 case, we use an hphp-multigrid preconditioner so that the polynomial degree of the coarsest grid level is reduced by one order, i.e. P=2\mathcal{P}=2 (the limitation on using P=1\mathcal{P}=1 is discussed at the beginning of § 5). The simulations are run on the Trillium distributed memory cluster of the Digital Research Alliance of Canada [74] and use up to 64 nodes with 192 CPU cores each (12,288 CPU cores in total).

l0l_0llCellsDoFs (Q2Q2Q_2Q_2)DoFs (Q3Q3Q_3Q_3)
404164k5.4M18M
4051.3M43M143M
40610.5M338M1.1B
185590k19M64M
1864.7M152M513M

Table 1. Number of cells and degrees of freedom (DoFs) for each refinement level employed in the scalability study of § 5.3.

The average number of linear solver iterations per Newton iteration for each refinement level are reported in Table 2; in all cases, the solver needed three non-linear Newton iterations. To evaluate the performance of the smoother preconditioner (i.e., relaxation scheme), results for both inverse diagonal (ID) and Additive Schwarz Method (ASM) options are presented. Furthermore, we consider a reference no-mortar case – same mesh configuration with a merged rotor-stator interface – to evaluate the impact that the mortar cells have on the solver convergence. The case with l0=40l_0 = 40 yields less iterations overall; this can be related to the fact that such discretization contains cells with a more uniform aspect ratio compared to when l0=18l_0 = 18. We conclude that the number of linear solver iterations is more affected by the aspect ratio of the cells than the presence of the mortar interface.

l0l_0llCellsQ2Q2Q_2Q_2 ID mmQ2Q2Q_2Q_2 ID nnQ2Q2Q_2Q_2 ASM mmQ2Q2Q_2Q_2 ASM nnQ3Q3Q_3Q_3 ID mmQ3Q3Q_3Q_3 ID nnQ3Q3Q_3Q_3 ASM mmQ3Q3Q_3Q_3 ASM nn
4164k211775181988
4051.3M231976202087
610.5M252175212177
185590k2533101027461115
64.7M2950101130631114

Table 2. Average number of linear iterations per Newton iteration for each refinement level obtained in the scalability study of § 5.3. Results are reported for two relaxation schemes: inverse diagonal (ID) and Additive Schwarz Method (ASM). For each scheme, the rotor-stator results (indicated by mm) are compared to a reference no-mortar case (indicated by nn) in which the rotor-stator interface is merged – leading to the same number of cells but less DoFs.

In terms of the smoother preconditioner type, ASM yields about a third less iterations than ID; however, it can take up to 3.4 times more computational time depending on the mesh refinement and the number of computer nodes. This result is expected considering the nature of these two relaxation schemes; while ID has a low cost for setup and application, ASM setup is more expensive while it usually leads to less iterations [53]. When comparing the mortar (mm) and the no-mortar (nn) cases, a similar number of iterations is shown for the ASM scheme. Meanwhile, a higher iteration count for the nn case is noticed for the ID scheme when l0=18l_0 = 18 when compared to the mortar results.

Figure 9 depicts the total wallclock time for the simulated cases. The strong scaling for the coarser meshes shows poorer results since very few cells are contained within each core, and hence communication significantly affects the wallclock time. Furthermore, the solution of a steady case can be harder compared to transient cases, since the flow needs to be solved at once. Significant improvement can be noticed for the two finest meshes, where the Q3Q3Q_3Q_3 curve shows an almost ideal strong scaling. The black arrows in the figure indicate the weak scalability, showing an increase in simulation time when both the mesh and the number of nodes are increased by a factor of 8. From 163.8k to 1.3M cells this increase is ∼41%\sim41\%, while from 1.3M to 10.5M cells is ∼105%\sim105\%. We highlight that weak scaling is also affected by the mismatch between the coarse meshes and the elevated number of nodes, where the communication cost predominates.

Image not extracted

Figure 9. Scalability study for a three-dimensional MMS problem considering the total wallclock time. The ideal strong scaling for each refinement level is indicated by dashed lines, while the black arrows indicate the weak scaling for the current case.

As noted in Figure 10, the mortar setup follows the same scalability trend as the total wallclock time. The reported values include time spent in creation of mortar operators, computation of boundary integrals, and update of the rotor mapping. Although the overall cells are distributed among processes considering load balancing, we assume that the cells at the rotor-stator interface and the remaining cells have the same weight. A question that arises is whether the load imbalance of mortar cells could have an effect on the efficiency of the solver. For such analysis, we first define an ideal workload as:

Image not extracted

Figure 10. Scalability study for a three-dimensional MMS problem considering the mortar setup time. The ideal strong scaling for each refinement level is indicated by dashed lines, while the black arrows indicate the weak scaling for the current case.

Lideal=∑inpminp(45)L_{\mathrm{ideal}} = \frac{\sum_{i}^{n_{\mathrm{p}}} m_i}{n_{\mathrm{p}}} \tag*{(45)}

in which npn_{\mathrm{p}} is the total number of processes and mim_i is the number of mortar cells at each process ii. The actual workload imbalance is computed as the ratio between the maximum number of cells in a process mmaxm_{\mathrm{max}} and the ideal workload:

L=mmaxLideal(46)L = \frac{m_{\mathrm{max}}}{L_{\mathrm{ideal}}} \tag*{(46)}

The computed workload imbalance is reported in Figure 11, as well as the percentage of time spent within the mortar setup. The latter varies around 20 to 50% of the total time, significantly contributing to the total wallclock time. Since different weights are not attributed to mortar cells during load balancing, workload imbalance increases significantly with the number of processes. Its deviation from the ideal ratio of 1 can be improved if we increase the weight of mortar cells, which consists in our next verification. For that, we vary the ratio between the mortar cell weight wmw_m and the regular cell weight wrw_r, monitoring the mortar workload imbalance factor.

Image not extracted

Figure 11. Mortar workload imbalance for a three-dimensional MMS problem. The solid lines represent the workload imbalance reported in left yy-axis, while the dashed lines indicate the percentage of mortar setup time with respect to the total simulation time and are reported in the right yy-axis.

For the case of 1.3M cells, Figure 12 depicts the mortar workload imbalance for various numbers of nodes (each node corresponds to 192 CPU cores). The workload stabilizes at approximately 2.5 when wm/wr>50w_m/w_r > 50; to analyze the impact that the mortar load balancing has on the simulation time, we choose wm/wr=20w_m/w_r = 20. When looking at the total wallclock time, especially in the Q3Q3Q_3Q_3 case, the cell weight ratio of 20 yields a good balance between improving the load imbalance without significantly increasing the computing time. Figure 13 compares the total wallclock time and the mortar workload imbalance for wm/wr=1w_m/w_r = 1 and wm/wr=20w_m/w_r = 20. Even though the load imbalance is significantly reduced – up to five times less, small variations are observed in the simulation time. In fact, performing the workload balancing requires slightly more time, confirming that it is an unnecessary step in the present mortar approach.

Figure 12. Mortar workload imbalance for a three-dimensional MMS problem with 1.3M cells and various number of computing nodes. The solid lines correspond to the left yy-axis, and the dashed lines represent the total wallclock time reported in the right yy-axis. Each node correspond to 192 CPU cores. The vertical dot-dashed gray line indicates the cell weight ratio that yields lower workload imbalance without significantly increasing the corresponding computing time.

Figure 13. Comparison of the total wallclock time (left yy-axis) and the mortar workload imbalance (right yy-axis) for (i) the regular case where no additional load balancing is performed and (ii) the case where load balancing is performed considering a cell weight ratio of wm/wr=20w_m/w_r = 20.

Two-dimensional Rushton mixer

To evaluate the MEM implementation in the context of an engineering application, we simulate a two-dimensional Rushton mixer proposed by Karim-25 ian and Sewerin [12]. The dimensions indicated in Figure 14a are detailed in Table 3. This example allows us to verify the solution against exist-

Two-dimensional Rushton mixer geometry and mesh with adaptive refinement

Figure 14. Two-dimensional Rushton mixer: (a) geometry and (b) mesh after adaptive refinement at final simulation time. In (b), the refinement level at the rotor-stator interface is kept the same throughout the simulation, and thus we ensure that the adaptive refinement does not create mismatching cell sizes at such boundaries.

ParameterDescriptionValue
DDTank diameter2 m
ddd_{d}Disc diameter0.5 m
did_{i}Impeller diameter0.89 m
θi\theta_{i}Blade opening angle7.5°
dintd_{int}Mortar interface diameterD/2D/2
θb\theta_{b}Baffle opening angleθi\theta_{i}
lbl_{b}Baffle lengthD/10D/10

Table 3. Dimensions of Rushton mixer tank example (Figure 14) presented in § 5.4.

ing finite-volume numerical results, as well as study the torque convergence with respect to the mesh refinement. We consider a kinematic viscosity of ν=10−2 m2/s\nu= 10^{-2}\ \mathrm{m^2/s}, a unit density (ρ=1 kg/m3\rho= 1\ \mathrm{kg/m^3}) and an impeller angular velocity of ω=2π rad/s\omega= 2\pi\ \mathrm{rad/s}; the Reynolds number is expressed as:

Re=ωdi22πν(47)Re = \frac{\omega d_i^2}{2\pi\nu} \tag*{(47)}

and thus, for the current example, Re=79Re = 79. The simulation is carried out for tend=20 st_{end} = 20\ \mathrm{s}, yielding twenty revolutions to ensure that the flow is fully established at the end. The adopted initial time step is Δt=10−4 s\Delta t = 10^{-4}\ \mathrm{s} and time step adaptation is enabled while the maximum allowed CFL is 1. The initial mesh contains Ne=4,732N_e = 4,732 biquadratic elements (70,422 DoFs); maintaining a fixed number of elements at the rotor-stator interface, an adaptive mesh refinement is also enabled (Figure 14b).

Unbaffled case: comparison against existing numerical results

To verify the accuracy of the mortar interface, we compare the result of an unbaffled Rushton mixer against the velocity profile reported by Karimian et al. [12]; the mixer dimensions are as indicated in Figure 14a. In such study, a mesh-tying method is used in a finite-volume context, where relative rotor-stator motion is represented by a rotating non-Galilean frame of reference. Figure 15a depicts the velocity profile along the xx-axis (at t=1 st = 1\,\mathrm{s}) for both the MEM solution and the finite-volume solution obtained in [12], where a good agreement is observed.

Figure 15. Two-dimensional Rushton mixer: (a) velocity profile at xx-axis at t=1 st = 1\,\mathrm{s}; unbaffled (b) velocity contour at tend=20 st_{end} = 20\,\mathrm{s}; baffled. (a) velocity profile for the unbaffled case, where the red dash-dotted line indicates the mortar interface position; (b) velocity field representation at tendt_{end} for the baffled case.

Baffled case: torque analysis

The simulation of the baffled geometry represented in Figure 14a yields the velocity field depicted in fig:15b at the final simulation time, when steady state is reached. We consider the baffled mixer configuration for a torque analysis study in which a uniform mesh refinement is employed. Figure 16 depicts the torque values at the impeller and the sum of torques at all boundaries for four refinement levels; l=0l = 0 corresponds to Ne=412N_e = 412 elements. While mesh convergence is observed for both measurements, it is also noticeable that the sum of torques converge to zero; the latter is an expected behavior given the laminar regime of the problem. Considering the biquadratic approximation of the velocity and pressure fields, we expect a second-order torque convergence rate. We can observe the convergence trend through the change in the steady-state measured values as the mesh is refined. For instance, the variation of the computed torque at the outer wall decreases from approximately 2.7% between l=0l = 0 and l=1l = 1 to 0.3% between l=2l = 2 and l=3l = 3.

Figure 16. Torque analysis for two-dimensional Rushton mixer presented in § 5.4: (a) torque at the impeller and (b) sum of torques at all boundaries.

Three-dimensional pitched-blade turbine (PBT) mixer

In this section we validate the matrix-free MEM solver with a three-dimensional pitched-blade turbine (PBT) mixer for which experimental results were reported by Blais et al. [2]. The geometry is shown in Figure 17, and the corresponding dimensions are indicated in Table 4. The fluid viscosity is varied so that we can impose different Reynolds number values; the latter is computed as:

Figure 17. Geometry of three-dimensional PBT mixer tank.

ParameterDescriptionValue
DtD_tTank diameter0.365 m
HtH_tTank heightDtD_t
CiC_iImpeller distance to bottomDt/4D_t/4
DiD_iImpeller diameterDt/3D_t/3
WiW_iImpeller blade widthDi/5D_i/5
TiT_iImpeller blade thicknessWi/10W_i/10
θ\thetaImpeller blade inclination45∘45^\circ
DsD_sShaft diameterWicos⁡(θ)W_i \cos(\theta)
DhD_hHub diameter1.4 Ds1.4\,D_s
HhH_hHub heightWiW_i
CbC_bBaffle distance to bottomCi/3C_i/3
SbS_bBaffle distance to outer tank wall19/20×Dt19/20 \times D_t
WbW_bBaffle widthDt/10D_t/10
TbT_bBaffle thicknessWb/3W_b/3

Table 4. Dimensions of PBT mixer tank (Figure 17).

Re=Di2Nν,(48)Re = \frac{D_i^2 N}{\nu}, \tag*{(48)}

where DiD_i is the impeller diameter in [m] as reported in Table 4, ν\nu is the kinematic viscosity in [m2^2/s], and NN is the impeller speed in [Hz]. Such rotation speed can also be represented in terms of the angular velocity ω=2πN\omega= 2\pi N [rad/s]. For the current problem, N=16N = 16 Hz, which yields a velocity at the tip of the impeller of utip≈2πu_{tip} \approx2\pi m/s. While the angular velocity is imposed at the impeller, a no-slip boundary condition is enforced on the outer cylinder, bottom wall, and baffles. A slip condition is applied to the top boundary to simulate a free fluid surface.

The experimental dataset [2] provides the power number NpN_p for Re∈[1,2×103]Re \in[1, 2 \times10^3] interval. The power number is defined as:

Np=2πTρN2Di5=PρN3Di5,(49)N_p = \frac{2\pi T}{\rho N^2 D_i^5} = \frac{P}{\rho N^3 D_i^5}, \tag*{(49)}

where TT is the torque at the impeller in [N · m] and P=2πNTP = 2\pi NT is the power consumption in [W].

The problem is simulated for tend=0.2t_{end} = 0.2 s, which yields 3.2 impeller revolutions. We consider a Newtonian fluid with unit density (ρ=1\rho= 1 kg/m3^3).

Similarly to the two-dimensional example from § 5.4, the maximum allowed CFL is 1, and the initial time step chosen is Δt=10−4 s\Delta t = 10^{-4}\,\mathrm{s}. The mesh discretization and the mortar interface placement are represented in Figure 18; the three refinement levels tested for different polynomial approximations are reported in Table 5. The cell size in the radial xx-yy plane and in the zz-direction must be constant at the mortar interface, which restricts further coarsening of the initial configuration. Nonetheless, the use of a matrix-free solver yields reasonable simulation times while the high-order polynomial approximation ensures sufficient accuracy of the solution – as discussed in the following section.

LevelCellsDoFs (Q2Q2Q_2Q_2)DoFs (Q3Q3Q_3Q_3)
032k1M3.6M
1252k8.4M28M
22M66M-

Table 5. Number of cells and degrees of freedom (DoFs) for each refinement level in the three-dimensional PBT example.

Mesh and mortar interface of the three-dimensional PBT mixer

Figure 18. (a) Mesh discretization of the three-dimensional PBT mixer. The coarsest level (herein represented) contains 31,459 cells. (b) Mortar interface (indicated in red) with diameter Dint=1.2DiD_{\mathrm{int}} = 1.2D_i, where DiD_i is the impeller diameter (Table 4). The discretization at the interface maintains a uniform cell size in both the xx-yy plane and zz-direction.

Mesh Sensitivity

To verify mesh convergence, we choose a Reynolds number of Re=200Re = 200, and hence ν=1.1842×10−3 m2/s\nu= 1.1842 \times10^{-3}\ \mathrm{m}^{2}/\mathrm{s}. For this case, we monitor the torque computed at the impeller as depicted in Figure 19a. Results are shown for Q2Q2Q_2Q_2 and Q3Q3Q_3Q_3 elements; we observe that the refinement level l=1l = 1 of the quadratic approximation yields converged torque values. Furthermore, the variation of the computed torque at the impeller from l=0l = 0 to l=1l = 1 is approximately 3.34%3.34\% for P=2\mathcal{P} = 2, and around 1.05%1.05\% when P=3\mathcal{P} = 3. This indicates that the coarsest mesh level with P=2\mathcal{P} = 2, even with only ≈31k\approx31\mathrm{k} cells, already yields fairly accurate results. Nevertheless, we choose the converged mesh configuration of Q2Q2Q_2Q_2, l=1l = 1 for the study presented in § 6.0.3.

Figure 19. Torque analysis at the impeller for three-dimensional PBT mixer.

It is noteworthy that the torque curves, as well as the energy balance curves reported in the upcoming § 6.0.2, depict an oscillatory pseudo-steady state profile. Part of this oscillation is due to the angular rotation speed imposed in the problem, and is asymptotically vanishing with time and mesh refinement. The second source of oscillation is the interaction between baffles and blades within the domain, causing the solution to converge to a sinusoidal-like signal that depends on the angle between the blades and the baffles.

The mesh sensitivity study is extended for Re=2000Re = 2000, and the torque evaluated at the impeller is shown in Figure 19b. The increased turbulence at this regime leads to more erratic oscillations in the torque curves; nonetheless, results for l=1l = 1 and l=2l = 2 for Q2Q2Q_2Q_2 elements are very close, and the same is observed for l=1l = 1 when P=3\mathcal{P} = 3. Considering the same solution parameters as in Re=200Re = 200, the results with the coarse mesh (l=0l = 0) for Re=2000Re = 2000 diverged at t≈0.05t \approx0.05 s. Consequently, choosing the Q2Q2Q_2Q_2, l=1l = 1 mesh discretization still yields converged results at the upper bound of the ReRe interval herein simulated.

Energy Analysis

Similarly to the energy analysis presented in [75] for a turbulent Taylor-Couette flow case, we evaluate the numerical dissipation inherent to the FEM model herein presented. For a domain Ω\Omega, the rate of change of kinetic energy in the system normalized by the density ρ\rho is given by [76]:

∫Ω∂∂t(12u2) dΩ=−∫Ω∇⋅(12u2u) dΩ−∫Ω∇⋅(p∗u) dΩ+∫Ωp∗(∇⋅u) dΩ−∫Ω∇(τ⋅u) dΩ+∫Ω(τ:∇u) dΩ,(50)\int_{\Omega} \frac{\partial}{\partial t}\left(\frac{1}{2}u^2\right)\,d\Omega= -\int_{\Omega} \nabla\cdot\left(\frac{1}{2}u^2\mathbf{u}\right)\,d\Omega- \int_{\Omega} \nabla\cdot\left(p^{*}\mathbf{u}\right)\,d\Omega+ \int_{\Omega} p^{*}(\nabla\cdot\mathbf{u})\,d\Omega- \int_{\Omega} \nabla(\boldsymbol{\tau}\cdot\mathbf{u})\,d\Omega+ \int_{\Omega}(\boldsymbol{\tau}:\nabla\mathbf{u})\,d\Omega, \tag*{(50)}

where uu is the magnitude of u\mathbf{u} and τ=−ν(∇u+∇uT)\boldsymbol{\tau} = -\nu(\nabla\mathbf{u} + \nabla\mathbf{u}^{\mathsf{T}}) is the deviatoric stress tensor. For a closed domain the first right-hand side (RHS) term is zero, and the third RHS term is also zero for an incompressible flow; we can integrate by parts the remaining RHS terms. Multiplying (49) again by the density so that we recover the physical meaning of the energy balance, the equation reduces to:

∫Ω∂∂t(12ρu2) dΩ⏟rate of kinetic energy=−∫∂Ωn⋅(ρ[τ+p∗]⋅u) d∂Ω⏟power at the wall+∫Ωρ(τ:∇u) dΩ⏟viscous dissipation.(51)\underbrace{\int_{\Omega}\frac{\partial}{\partial t}\left(\frac{1}{2}\rho u^2\right)\,d\Omega}_{\text{rate of kinetic energy}}=-\underbrace{\int_{\partial\Omega}\mathbf{n}\cdot\left(\rho[\boldsymbol{\tau}+p^*]\cdot\mathbf{u}\right)\,d\partial\Omega}_{\text{power at the wall}}+\underbrace{\int_{\Omega}\rho(\boldsymbol{\tau}:\nabla\mathbf{u})\,d\Omega}_{\text{viscous dissipation}}. \tag*{(51)}

The first RHS term represents the power applied to the wall, while the second term indicates the viscous dissipation within the domain. To evaluate the magnitude of the numerical dissipation, we rewrite the energy balance in terms of a normalized positive residual ϵn\epsilon_n, such that:

−ϵn=1T‾(∫Ω∂∂t(12u2) dΩ+∫∂Ωn⋅([τ+p∗]⋅u) d∂Ω+∫Ω(−τ:∇u) dΩ),(52)-\epsilon_n=\frac{1}{\overline{T}}\left(\int_{\Omega}\frac{\partial}{\partial t}\left(\frac{1}{2}u^2\right)\,d\Omega+\int_{\partial\Omega}\mathbf{n}\cdot\left([\boldsymbol{\tau}+p^*]\cdot\mathbf{u}\right)\,d\partial\Omega+\int_{\Omega}(-\boldsymbol{\tau}:\nabla\mathbf{u})\,d\Omega\right), \tag*{(52)}

in which T‾\overline{T} is the torque power computed at tendt_{end} for the finest mesh. Figure 20 depicts ϵn\epsilon_n over time; the observed numerical dissipation is around 2% for the coarsest Q2Q2Q_2Q_2 mesh, and it reaches approximately 1% for the finest meshes in both P=2,3\mathcal{P}=2,3 cases. At tend=0.2t_{end}=0.2 s, approximately 95% of the system’s kinetic energy is achieved. Additional tests for a case with Q1Q1Q_1Q_1 elements showed a numerical dissipation of ≈10%\approx10\%, which emphasizes the advantage of the high-order approximation.

Figure 20. Energy balance for three-dimensional PBT mixer at Re=200Re=200.

As observed in Figure 20, the residual numerical dissipation levels off at around 1% for both P=2P = 2 and P=3P = 3. To investigate the reason for this, additional tests were carried out by limiting the CFL to 0.5 and hence reducing the maximum time step by half. This change did indeed reduce the residual numerical dissipation, meaning that part of the remaining numerical error is related to the time discretization. Furthermore, the convergence of the individual terms in Eq. (52) might affect the overall convergence pattern. To this end, Figure 21 depicts the kinetic energy and the viscous dissipation rate. The mesh sensitivity of the two terms differs by one order of magnitude, highlighting how the behavior of each term contributes differently to the convergence of the energy balance residual.

Figure 21. Kinetic energy (left) and viscous dissipation rate (right) for three-dimensional PBT mixer at Re=200Re = 200.

The energy balance is also analyzed at Re=2000Re = 2000, as depicted in Figure 22. At this turbulence level, the coarsest mesh (l=0l = 0) is not able to properly represent the flow and hence the linear solver diverges quickly. Nonetheless, results for the other two mesh levels yield satisfactory numerical dissipation results: around 10% for l=1l = 1 and 3% for l=2l = 2 with Q2Q2Q_2Q_2 elements, and around 5% for l=1l = 1 and Q3Q3Q_3Q_3 elements. Moreover, the maximum CFL allowed in the simulation is 1, and tightening this restriction would reduce the time discretization error and hence the numerical dissipation (as noted for Re=200Re = 200). It is noteworthy that, while the mesh with refinement l=1l = 1 at Re=2000Re = 2000 is converged with respect to the torque metric, the energy residual remains more mesh sensitive.

Figure 22. Energy balance for three-dimensional PBT mixer at Re=2000Re = 2000.

Power Number Curve

Based on the mesh convergence study and the energy analysis, in this section we obtain the power number curve for the three-dimensional PBT mixer considering a mesh with Q2Q2Q_2Q_2 elements and refinement level l=1l = 1 (see Table 5). Twenty-four sample points were chosen in the Reynolds number interval [1,2×103][1, 2 \times10^3]; the simulations were carried out in the Trillium cluster from the Digital Research Alliance of Canada [74]. The simulation time decreased with the Reynolds number; for Re≥100Re \ge100, the average total wallclock time is 5h, while for Re<100Re < 100 the average running time is 8h.

The higher computational time required for low ReRe could be associated with the laminar nature of the problem. The impeller rotation is abruptly imposed at the first time step; when this boundary condition is associated with the low ReRe regime, which reaches a pseudo-steady state faster than at high ReRe, the linear solver requires more iterations to converge.

Figure 23 depicts the power number as a function of the Reynolds number for the experimental [2] and numerical (MEM) results. A good agreement is observed, especially at higher ReRe. Previous authors reported numerical results of the same problem configuration using a Nitsche Immersed Boundary (NIB) method [15], which are also included in Figure 23. The NIB and MEM curves are in very close agreement; the latter shows a slightly better estimation of the power number NpN_p at 1<Re<1001 < Re < 100 and at Re>500Re > 500 approximately. In terms of the setup and corresponding wallclock time for both methods, the NIB results reported in [15] considered a matrix-based solver with a Q1Q1Q_1Q_1 approximation. For a mesh with two million fluid cells and 120 thousand solid cells (i.e., impeller), corresponding to approximately 1.4M DoFs in total, the study was performed in three nodes of the Narval cluster (with 64 cores each). The comparison with the MEM model parameters herein adopted is summarized in Table 6. Even though the second-order approximation in the matrix-free solver increased significantly the number of degrees of freedom, the computational time was similar or even reduced in the MEM cases with Re>15Re > 15. Furthermore, the NIB simulation did not present an energy balance analysis, and hence quantification of the energy dissipation within the system was not verified.

Figure 23. Power curve for three-dimensional PBT mixer; comparison of the MEM model with experimental results [2] (represented by the various markers for each viscosity value) and with a Nitsche Immersed Boundary (NIB) method [15].

MethodSolverDegreeCellsDoFsCoresTime [h]
MEMmatrix-freeQ2Q2Q_2Q_2252k8M1×1921 \times1924.5 to 12
NIBmatrix-basedQ1Q1Q_1Q_12M(f)+120k(s)2\mathrm{M}^{(f)}+120\mathrm{k}^{(s)}1.4M3×643 \times647.5 to 12.5

Table 6. Setup configuration and corresponding computational time for the three-dimensional PBT mixer example considering the Mortar Element Method (MEM) and the Nitsche Immersed Boundary (NIB) method results in [2]. The superscripts in the NIB case indicate the number of cells in the fluid (ff) and solid (ss) domains.

Conclusions and further research

A mortar element method within a matrix-free high-order framework was herein described and employed to simulate rotating mixing devices. The fluid domain was discretized using a CG formulation, while the mortar interface was represented with a DG framework. Continuity at such interface was imposed through a weak formulation of boundary integrals and employing a Symmetric Interior Penalty Galerkin method. An Arbitrary Lagrangian-Eulerian approach was used to account for the relative motion of the rotor and stator domains.

Two-dimensional examples were used to verify the convergence rate of the MEM matrix-free model. Both steady and transient cases yielded optimal rates for velocity and pressure fields, recovering the same order as in a standard (non-mortar) case. Regardless of the rotation angle, the geometric misalignment generated at the interface does not lead to loss of convergence properties. A three-dimensional steady case was used to evaluate the scalability of the model. When the problem is sufficiently large for the number of computing cores used, a good strong scalability is achieved. Moreover, improving the load balancing of the mortar cells did not decrease computing time significantly.

A two-dimensional Rushton mixer case validated the matrix-free MEM model against existing numerical results, and the presented torque analysis showed the expected convergence. Furthermore, a three-dimensional PBT mixer example validated the model against experimental results, where the MEM power number curve showed great agreement over the ReRe experimental range. An energy analysis was also performed, in which the numerical dissipation introduced by the model was quantified. The converged meshes yielded ∼1%\sim1\% numerical dissipation at Re=200Re = 200 and 10%10\% at Re=2000Re = 2000, and thus the implicit Large Eddy Simulation (LES) model herein employed is able to very accurately solve transitional/early turbulent flows.

Future work encompasses the simulation of other devices with more complex geometries and/or flow patterns to showcase the suitability of the developed matrix-free MEM model to a wide range of engineering applications.

A current limitation of the model is the restriction of equal-sized cells at the interface. It could be advantageous, however, to allow an unstructured mesh as well as a varying interface radius so that meshing of more complex geometries is simplified; this will be addressed in future model extensions. Upcoming studies also comprise the extension of the model to couple other physics such as particle transport within a fluid.

Acknowledgments

The authors acknowledge collaboration with the deal.II community, and the technical support and computing time provided by the Digital Research Alliance of Canada and Calcul Québec. The authors also acknowledge the financial support of Médicament Québec and the Quebec Consortium for Drug Discovery (CQDM) through the ARENA program, managed by the CQDM. Bruno Blais acknowledges financial support from the Natural Sciences and Engineering Research Council of Canada (NSERC) through the RGPIN-2020-04510 Discovery Grant and the funding from the Multiphysics Multiphase Intensification Automatization Workbench (MMIAOW) Canadian Research Chair Level 2 in computer-assisted design and scale-up of alternative energy vectors for sustainable chemical processes (CRC-2022-00340).

References

  1. [1]P. Oxley, C. Brechtelsbauer, F. Ricard, N. Lewis, C. Ramshaw, Evaluation of spinning disk reactor technology for the manufacture of pharmaceuticals, Industrial & engineering chemistry research 39 (7) (2000) 2175–2182. doi:10.1021/ie990869u.DOI
  2. [2]B. Blais, M. Lassaigne, C. Goniya, L. Fradette, F. Bertrand, A. semi-implicit immersed boundary method and its application to viscous mixing, Computers & Chemical Engineering 85 (2016) 136–146. doi:10.1016/j.compchemeng.2015.10.019.DOI
  3. [3]F. Zahle, N. N. Sørensen, J. Johansen, Wind turbine rotor-tower interaction using an incompressible overset grid method, Wind Energy: An International Journal for Progress and Applications in Wind Power Conversion Technology 12 (6) (2009) 594–619. doi:10.1002/we.327.DOI
  4. [4]C. Rauwendaal, Mixing in polymer processing, Vol. 23, CRC Press, 1991.
  5. [5]D. Zou, J. Li, J. Deng, Y. Jin, M. Chen, Continuous preparation of size-controllable BaTiO3 nanoparticles in a rotor–stator spinning disk reactor, Industrial & Engineering Chemistry Research 60 (40) (2021) 14511–14518. doi:10.1021/acs.iecr.1c02152.DOI
  6. [6]P. J. Cullen, Food mixing: Principles and applications, John Wiley & Sons, 2009.
  7. [7]C. Mavriplis, Nonconforming discretizations and a posteriori error estimators for adaptive spectral element techniques, Ph.D. thesis, Massachusetts Institute of Technology (1989).DOI
  8. [8]C. Bernardi, Y. Maday, A. T. Patera, Domain decomposition by the mortar element method, in: Asymptotic and numerical methods for partial differential equations with critical parameters, Springer, 1993, pp. 269–286.DOI
  9. [9]F. B. Belgacem, Y. Maday, The mortar element method for three dimensional finite elements, ESAIM: Mathematical Modelling and Numerical Analysis 31 (2) (1997) 289–302. doi:10.1051/m2an/1997310202891.
  10. [10]B. E. Merrill, Y. T. Peet, P. F. Fischer, J. W. Lottes, A spectrally accurate method for overlapping grid solution of incompressible navier–stokes equations, Journal of Computational Physics 307 (2016) 60–93. doi:10.1016/j.jcp.2015.11.057.DOI
  11. [11]A. Hegazy, D. Shaver, R. Uddin, A. Novak, Immersed Boundary Methods for the NekRS Spectral Element Navier-Stokes Solver, Available at SSRN 6834480.
  12. [12]K. Karimian, F. Sewerin, A finite volume mesh-tying method for computing flows in rotor-stator configurations, Journal of Computational Physics 531 (2025) 113863. doi:10.1016/j.jcp.2025.113863.DOI
  13. [13]J. B. Joshi, N. K. Nere, C. V. Rane, B. Murthy, C. S. Mathpati, A. W. Patwardhan, V. V. Ranade, CFD simulation of stirred tanks: Comparison of turbulence models. Part I: Radial flow impellers, The Canadian Journal of Chemical Engineering 89 (1) (2011) 23–82. doi:10.1002/cjce.20446.DOI
  14. [14]E. Burman, M. A. Fernández, Explicit strategies for incompressible fluid-structure interaction problems: Nitsche type mortaring versus Robin–Robin coupling, International Journal for Numerical Methods in Engineering 97 (10) (2014) 739–758. doi:10.1002/nme.4607.DOI
  15. [15]J. Joachim, C.-A. Daunais, V. Bibeau, L. Heltai, B. Blais, A parallel and adaptive Nitsche immersed boundary method to simulate viscous mixing, Journal of Computational Physics 488 (2023) 112189. doi:10.1016/j.jcp.2023.112189.DOI
  16. [16]L. Barbeau, S. Golshan, J. Deng, S. Étienne, C. Béguin, B. Blais, High-order moving immersed boundary and its application to a resolved CFD-DEM model, Computers & Fluids 268 (2024) 106094. doi:10.1016/j.compfluid.2023.106094.DOI
  17. [17]F. B. Belgacem, Y. Maday, Non-conforming spectral method for second order elliptic problems in 3d, East-West J. Numer. Math 1 (4) (1993) 235–251.
  18. [18]H. Feng, C. Mavriplis, R. Van der Feng, R. Biswas, Parallel 3D mortar element method for adaptive nonconforming meshes, Journal of Scientific Computing 27 (1) (2006) 231–243. doi:10.1007/s10915-005-9026-z.DOI
  19. [28]B. Yang, T. A. Laursen, X. Meng, Two dimensional mortar contact methods for large deformation frictional sliding, International Journal for Numerical Methods in Engineering 62 (9) (2005) 1183–1225. doi:10.1002/nme.1222.
  20. [29]F. P. Baaijens, A fictitious domain/mortar element method for fluid–structure interaction, International Journal for Numerical Methods in Fluids 35 (7) (2001) 743–761. doi:10.1002/1097-0363(20010415)35:7<743::AID-FLD109>3.0.CO;2-A.doi.org/10.1002/1097-0363(20010415)35:7<743::aid-fld109>3.0.co;2-a
  21. [30]N. Hagmeyer, M. Mayr, A. Popp, A fully coupled regularized mortar-type finite element approach for embedding one-dimensional fibers into three-dimensional fluid flow, International Journal for Numerical Methods in Engineering 125 (8) (2024) e7435. doi:10.1002/nme.7435.DOI
  22. [31]A. Popp, M. Gitterle, M. W. Gee, W. A. Wall, A dual mortar approach for 3D finite deformation contact with consistent linearization, International Journal for Numerical Methods in Engineering 83 (11) (2010) 1428–1465. doi:10.1002/nme.2866.DOI
  23. [32]A. Popp, B. I. Wohlmuth, M. W. Gee, W. A. Wall, Dual quadratic mortar finite element methods for 3D finite deformation contact, SIAM Journal on Scientific Computing 34 (4) (2012) B421–B446. doi:10.1137/110848190.DOI
  24. [33]J. D. Sanders, T. A. Laursen, M. A. Puso, A Nitsche embedded mesh method, Computational Mechanics 49 (2) (2012) 243–257. doi:10.1007/s00466-011-0641-2.DOI
  25. [34]R. Stenberg, Mortaring by a method of JA Nitsche, Computational Mechanics – New Trends and Applications. CIMNE, Barcelona (1998).
  26. [35]T. Gustafsson, R. Stenberg, J. Videman, Error analysis of Nitsche’s mortar method, Numerische Mathematik 142 (4) (2019) 973–994. doi:10.1007/s00211-019-01039-5.DOI
  27. [36]T. Gustafsson, R. Stenberg, J. Videman, On Nitsche’s method for elastic contact problems, SIAM Journal on Scientific Computing 42 (2) (2020) B425–B446. doi:10.1137/19M1246869.DOI
  28. [37]A. Bansal, N. A. Barnafi, D. N. Pandey, R. Ruiz-Baier, A. Nitsche method for Navier–Stokes/generalized poroelasticity interface problems, arXiv preprint arXiv:2508.09388 (2025).arxiv.org/abs/2508.09388
  29. [38]L. Formaggia, F. Gatti, S. Zonca, An XFEM/DG approach for fluid-structure interaction problems with contact, Applications of Mathematics 66 (2) (2021) 183–211. doi:10.21136/AM.2021.0310-19.DOI
  30. [39]S. Bertoluzza, M. Pennacchio, C. Prud’Homme, A. Samake, Sub-structuring preconditioners for h-p mortar FEM, ESAIM: Mathematical Modelling and Numerical Analysis 50 (4) (2016) 1057–1082. doi:10.1051/m2an/2015065.
  31. [40]J. Dürrwächter, M. Kurz, P. Kopper, D. Kempf, C.-D. Munz, A. Beck, An efficient sliding mesh interface method for high-order discontinuous Galerkin schemes, Computers & Fluids 217 (2021) 104825. doi:10.1016/j.compfluid.2020.104825.DOI
  32. [41]M. A. Kopera, F. X. Giraldo, Mass conservation of the unified continuous and discontinuous element-based Galerkin methods on dynamically adaptive grids with application to atmospheric simulations, Journal of Computational Physics 297 (2015) 90–103. doi:10.1016/j.jcp.2015.05.010.DOI
  33. [42]E. Ferrer, R. H. Willden, A high order discontinuous Galerkin–Fourier incompressible 3D Navier–Stokes solver with rotating sliding meshes, Journal of Computational Physics 231 (21) (2012) 7037–7056. doi:10.1016/j.jcp.2012.04.039.DOI
  34. [43]E. Laughton, Enabling efficient high-fidelity computational fluid dynamics (CFD) for moving geometries, Ph.D. thesis, University of Exeter (2024).
  35. [44]C. W. Hirt, A. A. Amsden, J. Cook, An arbitrary Lagrangian-Eulerian computing method for all flow speeds, Journal of computational physics 14 (3) (1974) 227–253. doi:10.1016/0021-9991(74)90051-5.DOI
  36. [45]C. S. Venkatasubban, A new finite element formulation for ALE (Arbitrary Lagrangian Eulerian) compressible fluid mechanics, International journal of engineering science 33 (12) (1995) 1743–1762. doi:10.1016/0020-7225(95)00021-O.DOI
  37. [55]Z. J. Wang, K. Fidkowski, R. Abgrall, F. Bassi, D. Caraeni, A. Cary, H. Deconinck, R. Hartmann, K. Hillewaert, H. T. Huynh, et al., High-order CFD methods: current status and perspective, International Journal for Numerical Methods in Fluids 72 (8) (2013) 811–845. doi:10.1002/fld.3767.DOI
  38. [56]B. Zhang, C. Liang, A simple, efficient, and high-order accurate curved sliding-mesh interface approach to spectral difference method on coupled rotating and stationary domains, Journal of Computational Physics 295 (2015) 147–160. doi:10.1016/j.jcp.2015.04.006.DOI
  39. [57]A. Alphonius, L. Barbeau, B. Blais, O. Gaboriault, O. Guévremont, J. Lamouche, P. Laurentin, O. Marquis, P. Munch, V. Oliveira Ferreira, et al., Lethe 1.0: An open-source high-performance and high-order computational fluid dynamics software for single and multiphase flows, Computer Physics Communications (2025) 109880.DOI
  40. [58]D. Arndt, W. Bangerth, D. Davydov, T. Heister, L. Heltai, M. Kronbichler, M. Maier, J.-P. Pelteret, B. Turcksin, D. Wells, The deal.II finite element library: Design, features, and insights, Computers & Mathematics with Applications 81 (2021) 407–422. doi:10.1016/j.camwa.2020.02.022.DOI
  41. [59]D. Arndt, W. Bangerth, M. Bergbauer, B. Blais, M. Fehling, R. Gassmöller, T. Heister, L. Heltai, M. Kronbichler, M. Maier, et al., The deal. ii library, version 9.7, Journal of Numerical Mathematics 33 (4) (2025) 403–415. doi:10.1515/jnma-2025-0115.DOI
  42. [60]M. G. Larson, F. Bengzon, The finite element method: theory, implementation, and applications, Vol. 10, Springer Science & Business Media, 2013.DOI
  43. [61]P. Munch, T. Heister, L. Prieto Saavedra, M. Kronbichler, Efficient distributed matrix-free multigrid methods on locally refined meshes for FEM computations, ACM Transactions on Parallel Computing 10 (1) (2023) 1–38. doi:10.1145/3580314.
  44. [62]N. Fehn, W. A. Wall, M. Kronbichler, Efficiency of high-performance discontinuous Galerkin spectral element methods for under-resolved turbulent incompressible flows, International Journal for Numerical Methods in Fluids 88 (1) (2018) 32–54. doi:10.1002/fld.4511.DOI
  45. [63]N. Fehn, Robust and efficient discontinuous Galerkin methods for incompressible flows, Ph.D. thesis, Technische Universität München (2021).
  46. [64]D. N. Arnold, An interior penalty finite element method with discontinuous elements, SIAM journal on numerical analysis 19 (4) (1982) 742–760.DOI
  47. [65]D. N. Arnold, F. Brezzi, B. Cockburn, L. D. Marini, Unified analysis of discontinuous galerkin methods for elliptic problems, SIAM journal on numerical analysis 39 (5) (2002) 1749–1779. doi:10.1137/S0036142901384162.DOI
  48. [66]K. Hillewaert, Development of the discontinuous Galerkin method for high-resolution, large scale CFD and acoustics in industrial geometries, Ph.D. thesis, Université de Louvain (2013).
  49. [67]T. E. Tezduyar, Stabilized finite element formulations for incompressible flow computations, Advances in Applied Mechanics 28 (1991) 1–44. doi:10.1016/S0065-2156(08)70153-4.DOI
  50. [68]C. Burstedde, L. C. Wilcox, O. Ghattas, p4est: Scalable algorithms for parallel adaptive mesh refinement on forests of octrees, SIAM Journal on Scientific Computing 33 (3) (2011) 1103–1133. doi:10.1137/100791634.DOI
  51. [69]M. A. Heroux, R. A. Bartlett, V. E. Howle, R. J. Hoekstra, J. J. Hu, T. G. Kolda, R. B. Lehoucq, K. R. Long, R. P. Pawlowski, E. T. Phipps, et al., An overview of the trilinos project, ACM Transactions on Mathematical Software (TOMS) 31 (3) (2005) 397–423. doi:10.1145/1089014.1089021.DOI
  52. [70]J. Heinz, P. Munch, M. Kaltenbacher, High-order non-conforming discontinuous Galerkin methods for the acoustic conservation equations, International Journal for Numerical Methods in Engineering 124 (9) (2023) 2034–2049. doi:10.1002/nme.7199.DOI
  53. [71]M. Bergbauer, P. Munch, W. A. Wall, M. Kronbichler, High-performance matrix-free unfitted finite element operator evaluation, SIAM Journal on Scientific Computing 47 (3) (2025) B665–B689. doi:10.1137/24M1653689.DOI
  54. [72]P. Wesseling, C. W. Oosterlee, Geometric multigrid with applications to computational fluid dynamics, Journal of Computational and Applied Mathematics 128 (1-2) (2001) 311–334. doi:10.1016/S0377-0427(00)00517-3.DOI
  55. [73]O. C. Zienkiewicz, R. L. Taylor, P. Nithiarasu, J. Zhu, The finite element method, Vol. 3, Elsevier, 1977.
  56. [74]Digital Research Alliance of Canada, Trillium documentation, https://docs.alliancecan.ca/wiki/Trillium, Last accessed 17 December 2025 (2025).
  57. [75]L. Prieto Saavedra, J. Archambault, P. Munch, B. Blais, An implicit large-eddy simulation study of the turbulent Taylor-Couette flow with an inner rotating cylinder, Journal of Turbulence 26 (6-7) (2025) 219–233. doi:10.1080/14685248.2025.2524336.DOI
  58. [76]R. B. Bird, W. E. Stewart, E. N. Lightfoot, Transport Phenomena, John Wiley & Sons, 2002.

Paper details

vlnc:W_6b8mvc34
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

Not recorded for this paper

View statement
Looking for related papers…
Contents