Advertisement

AMGCL —A C++ library for efficient solution of large sparse linear systems

      Highlights

      • C++ library for solving large sparse linear systems with algebraic multigrid.
      • Supports MPI, OpenMP, CUDA, OpenCL.
      • MIT license.

      Abstract

      AMGCL is a header-only C++ library for the solution of large sparse linear systems with algebraic multigrid. The method may be used as a black-box solver for computational problems in various fields, since it does not require any information about the underlying geometry. AMGCL provides an efficient, flexible, and extensible implementation of several iterative solvers and preconditioners on top of different backends allowing the acceleration of the solution with the help of OpenMP, OpenCL, or CUDA technologies. Most algorithms have both shared memory and distributed memory implementations. The library is published under a permissive MIT license.

      MSC

      Keywords

      Code metadata
      Tabled 1
      Current code version1.3.99
      Permanent link to code/repository used for this code versionhttps://github.com/SoftwareImpacts/SIMPAC-2020-51
      Permanent link to Reproducible Capsulehttps://codeocean.com/capsule/4394229/tree/v1
      Legal Code LicenseMIT
      Code versioning system usedgit
      Software code languages, tools, and services usedC++, MPI, OpenMP, OpenCL, CUDA
      Compilation requirements, operating environmentsC++11 compiler, backend requirements
      Link to developer documentation/manualhttps://amgcl.readthedocs.io
      Support email for questions[email protected]

      1. Introduction

      Most of the numerical simulation problems today involve solution of large sparse linear systems obtained from discretization of partial differential equations on either structured or unstructured meshes. The combination of a Krylov subspace method with algebraic multigrid (AMG) as a preconditioner is considered to be one the most effective choices for solution of such systems [
      • Brandt A.
      • McCoruick S.
      • Huge J.
      Algebraic multigrid (AMG) for sparse matrix equations.
      ,
      • Ruge J.W.
      • Stüben K.
      Algebraic multigrid.
      ,
      • Trottenberg U.
      • Oosterlee C.
      • Schüller A.
      Multigrid.
      ]. The AMG can be used as a black box solver for various computational problems, since it does not require any information about the underlying geometry, and is known to be robust and scalable [
      • Cleary A.J.
      • Falgout R.D.
      • Henson V.E.
      • Jones J.E.
      • Manteuffel T.A.
      • McCormick S.F.
      • Miranda G.N.
      • Ruge J.W.
      Robustness and scalability of algebraic multigrid.
      ].
      AMGCL is a header-only library implementing multiple Krylov subspace iterative solvers preconditioned with the algebraic multigrid method [
      • Demidov D.
      AMGCL: An efficient, flexible, and extensible algebraic multigrid implementation.
      ]. It has a minimal set of dependencies and provides both shared memory and distributed versions of the algorithms. The multigrid hierarchy is constructed using builtin data structures and then transferred into one of the provided backends. This allows for transparent acceleration of the solution phase with help of OpenMP, OpenCL, or CUDA technologies. The library users may also provide their own backends which enables tight integration between AMGCL and the user code.
      Although the initial focus of the library was the implementation of the algebraic multigrid, its modular architecture allowed to provide more specialized preconditioners, such as CPR [
      • Stüben K.
      • Clees T.
      • Klie H.
      • Lu B.
      • Wheeler M.F.
      • et al.
      Algebraic multigrid methods (AMG) for the efficient solution of fully implicit formulations in reservoir simulation.
      ] or Schur pressure correction [
      • Saleri F.
      • Veneziani A.
      Pressure correction algebraic splitting methods for the incompressible Navier–Stokes equations.
      ]. The multigrid relaxation components may be used as single level preconditioners.

      2. AMGCL design principles

      The main drivers behind AMGCL design are usability, efficiency, and extensibility. The design principles that help to achieve the goals are described below.
      Figure thumbnail fx1004
      Preference for free functions as opposed to member functions [
      • Meyers S.
      Effective C++: 55 Specific Ways to Improve your Programs and Designs.
      ], combined with partial template specialization allows to extend the library operations onto user-defined datatypes and to introduce new algorithmic components when required.
      The backend system of the library allows expressing the algorithms such as Krylov iterative solvers or multigrid relaxation methods in terms of generic parallel primitives which facilitates transparent acceleration of the solution phase with OpenMP, OpenCL, or CUDA technologies. The value types are one level below the backends: AMGCL supports systems with scalar, complex, or block value types both in single and double precision. Arithmetic operations necessary for the library work may also be extended onto the user-defined types using template specialization.

      3. Impact

      AMGCL demonstrates performance comparable [
      • Demidov D.
      AMGCL: An efficient, flexible, and extensible algebraic multigrid implementation.
      ,
      • Demidov D.
      • Mu L.
      • Wang B.
      Accelerating linear solvers for large-scale Stokes problems with c++ metaprogramming.
      ] with popular software packages, such as PETSC [
      • Balay S.
      • Abhyankar S.
      • Adams M.F.
      • Brown J.
      • Brune P.
      • Buschelman K.
      • Dalcin L.
      • Eijkhout V.
      • Gropp W.D.
      • Kaushik D.
      • Knepley M.G.
      • McInnes L.C.
      • Rupp K.
      • Smith B.F.
      • Zampini S.
      • Zhang H.
      • Zhang H.
      PETSc Users Manual Tech. Rep. ANL-95/11 - Revision 3.7.
      ], Trilinos [
      • Heroux M.A.
      • Bartlett R.A.
      • Howle V.E.
      • Hoekstra R.J.
      • Hu J.J.
      • Kolda T.G.
      • Lehoucq R.B.
      • Long K.R.
      • Pawlowski R.P.
      • Phipps E.T.
      • et al.
      An overview of the Trilinos project.
      ], CUSP [
      • Dalton S.
      • Bell N.
      • Olson L.
      • Garland M.
      Cusp: Generic parallel algorithms for sparse matrix and graph computations.
      ], or PARDISO [
      • Schenk O.
      • Gärtner K.
      • Fichtner W.
      • Stricker A.
      PARDISO: a high-performance serial and parallel sparse linear solver in semiconductor device simulation.
      ]. The fact that the library allows the user to easily switch to mixed precision approach or use block-valued system formulation may yield significant savings in both computation time and memory requirements [
      • Demidov D.
      • Mu L.
      • Wang B.
      Accelerating linear solvers for large-scale Stokes problems with c++ metaprogramming.
      ]. Another advantage of AMGCL is that it has less steep learning curve and lower entry cost than such large-scale packages as PETSC or Trilinos, and is published under permissive MIT license which allows for commercial use of the library.
      AMGCL is used as a default solver in the Kratos Multi-Physics framework [
      • Dadvand P.
      • Rossi R.
      • Oñate E.
      An object-oriented environment for developing finite element codes for multi-disciplinary applications.
      ] developed at CIMNE, Barcelona. Linear solvers based on AMGCL are also provided as part of the MATLAB Reservoir Simulation Toolbox (MRST) [
      • Lie K.-A.
      An Introduction to Reservoir Simulation using MATLAB/GNU Octave: User Guide for the MATLAB Reservoir Simulation Toolbox (MRST).
      ] developed by the Computational Geosciences group in the Department of Mathematics and Cybernetics at SINTEF Digital. The library was used in research on subsurface flow [
      • Shangaraeva A.
      • Shevchenko D.
      Speed up of the oil saturation numerical algorithm for the plane-parallel filtration.
      ,
      • Goncharova G.
      • Khramchenkov M.
      Mathematical model of hydraulic fracturing of a bed.
      ,
      • Cotela Dalmau J.
      • Rossi R.
      • Larese De Tetto A.
      Simulation of two-and three-dimensional viscoplastic flows using adaptive mesh refinement.
      ,
      • Longva A.B.
      Finite Element Solutions to the Wave Equation in Non-Convex Domains - A Relaxation of the CFL Condition in the Presence of Local Mesh Refinement.
      ,
      • Khramchenkov E.
      • Khramchenkov M.
      Numerical model of two-phase flow in dissolvable porous media and simulation of reservoir acidizing.
      ,
      • Krogstad S.
      • Nilsen H.
      • Møyner O.
      • Rasmussen A.
      Well control optimization of the OLYMPUS case using MRST and OPM.
      ,
      • Møyner O.
      • Tchelepi H.A.
      • et al.
      A mass-conservative sequential implicit multiscale method for isothermal equation-of-state compositional problems.
      ,
      • Klemetsdal Ø.S.
      • Møyner O.
      • Lie K.-A.
      • et al.
      Robust nonlinear newton solver with adaptive interface-localized trust regions.
      ,
      • Khramchenkov M.
      • Khramchenkov E.
      • Usmanov R.
      Non-linear equations of mechanics of swelling and metamorphic processes.
      ,
      • Zakirov T.
      • Galeev A.
      Absolute permeability calculations in micro-computed tomography models of sandstones by Navier-Stokes and lattice Boltzmann equations.
      ,
      • Demidov D.
      • Rossi R.
      Subdomain deflation combined with local AMG: A case study using AMGCL library.
      ,
      • Hashemi M.R.
      • Ryzhakov P.B.
      • Rossi R.
      An enriched finite element/level-set method for simulating two-phase incompressible fluid flows with surface tension.
      ,
      • Klemetsdal Ø.
      • Moncorgé A.
      • Moyner O.
      • Lie K.
      Additive Schwarz preconditioned exact Newton method as a nonlinear preconditioner for multiphase porous media flow.
      ,
      • Li D.
      • Xu K.
      • Harris J.M.
      • Darve E.
      Coupled time-lapse full-waveform inversion for subsurface flow problems using intrusive automatic differentiation.
      ,
      • Nilsen H.
      • Moncorge A.
      • Bao K.
      • Møyner O.
      • Lie K.
      • Brodtkorb A.
      Comparison between algebraic multigrid and multilevel multiscale methods for reservoir simulation.
      ,
      • Pinzinger R.
      • Blankenburg R.
      Speeding up the computation of the transient Richards’ equation with AMGCL.
      ,
      • Rasmussen A.F.
      • Sandve T.H.
      • Bao K.
      • Lauser A.
      • Hove J.
      • Skaflestad B.
      • Klöfkorn R.
      • Blatt M.
      • Rustad A.B.
      • Sævareid O.
      • Lie K.A.
      • Thune A.
      The open porous media flow reservoir simulator.
      ,
      • Sbai M.A.
      • Larabi A.
      On solving groundwater flow and transport models with algebraic multigrid preconditioning.
      ,
      • Yakirevich A.
      Water flow, solute and heat transfer in groundwater.
      ], fluid simulation [
      • Isaev S.
      • Leontiev A.
      • Chudnovsky Y.
      • Nikushchenko D.
      • Popov I.
      • Sudakov A.
      Simulation of vortex heat transfer enhancement in the turbulent water flow in the narrow plane-parallel channel with an inclined oval-trench dimple of fixed depth and spot area.
      ,
      • Isaev S.
      Thermal-hydrodynamic design of energy-efficient surfaces with inclined oval-trench vortex generators.
      ,
      • Isaev S.
      • Gritckevich M.
      • Leontiev A.
      • Milman O.
      • Nikushchenko D.
      NT vortex enhancement of heat transfer and flow in the narrow channel with a dense packing of inclined one-row oval-trench dimples.
      ,
      • Isaev S.
      • Leontiev A.
      • Milman O.
      • Popov I.
      • Sudakov A.
      Influence of the depth of single-row oval-trench dimples inclined to laminar air flow on heat transfer enhancement in a narrow micro-channel.
      ,
      • Ryzhakov P.B.
      • et al.
      On the relevance of accounting for uid-structure interaction e ects in the numerical studies of type b aortic dissection.
      ,

      J. Al-Salami, C. Hu, M.M. Kamra, K. Hanada, Magnetic induction and electric potential smoothed particle magnetohydrodynamics for incompressible flows, Internat. J. Numer. Methods Fluids n/a (n/a). http://dx.doi.org/10.1002/fld.4906.

      ,
      • Ando R.
      • Batty C.
      A practical octree liquid simulator with adaptive surface resolution.
      ,
      • Fang Y.
      • Qu Z.
      • Li M.
      • Zhang X.
      • Zhu Y.
      • Aanjaneya M.
      • Jiang C.
      IQ-MPM: an interface quadrature material point method for non-sticky strongly two-way coupled nonlinear solids and fluids.
      ,
      • Gagniere S.
      • Hyde D.
      • Marquez-Razon A.
      • Jiang C.
      • Ge Z.
      • Han X.
      • Guo Q.
      • Teran J.
      A hybrid Lagrangian/Eulerian collocated velocity advection and projection method for fluid simulation.
      ], air flow [
      • Isaev S.
      • Schelchkov A.
      • Leontiev A.
      • Gortyshov Y.F.
      • Baranov P.
      • Popov I.
      Vortex heat transfer enhancement in the narrow plane-parallel channel with the oval-trench dimple of fixed depth and spot area.
      ,
      • Isaev S.
      • Baranov P.
      • Popov I.
      • Sudakov A.
      • Usachov A.
      • Guvernyuk S.
      • Sinyavin A.
      • Chylunin A.
      • Mazo A.
      • Kalinin E.
      Ensuring safe descend of reusable rocket stages – Numerical simulation and experiments on subsonic turbulent air flow around a semi-circular cylinder at zero angle of attack and moderate Reynolds number.
      ,
      • Isaev S.
      • Leont’ev A.
      • Mil’man O.
      • Sudakov A.
      • Usachov A.
      • Gul’tsova M.
      Intensification of heat exchange in laminar vortex air flow in a narrow channel with a row of inclined oval trenches.
      ,
      • Isaev S.
      • Baranov P.
      • Popov I.
      • Sudakov A.
      • Usachov A.
      • Guvernyuk S.
      • Sinyavin A.
      • Chulyunin A.
      • Mazo A.
      • Demidov D.
      • et al.
      Numerical simulation and experiments on turbulent air flow around the semi-circular profile at zero angle of attack and moderate Reynolds number.
      ,
      • Isaev S.
      • Leontiev A.
      • Milman O.
      • Nikushchenko D.
      • Sudakov A.
      The effect of anomalous enhancement of vortex heat transfer in the separated flow in inclined oval-trench dimple and on the structured surfaces.
      ,
      • Isaev S.
      • Leontiev A.
      • Milman O.
      • Nikushchenko D.
      • Egorova A.
      Energy-efficient surface of air capacitors with inclined single-row oval-trench dimples and protrusions.
      ], machine learning [
      • Parag T.
      • Plaza S.
      • Scheffer L.
      Small sample learning of superpixel classifiers for EM segmentation.
      ,
      • Xu K.
      • Tartakovsky A.M.
      • Burghardt J.
      • Darve E.
      Inverse modeling of viscoelasticity materials using physics constrained learning.
      ,
      • Xu K.
      • Darve E.
      Physics constrained learning for data-driven inverse modeling from sparse observations.
      ], micromagnetics [
      • Fischbacher J.
      • Kovacs A.
      • Oezelt H.
      • Schrefl T.
      • Exl L.
      • Fidler J.
      • Suess D.
      • Sakuma N.
      • Yano M.
      • Kato A.
      • et al.
      Nonlinear conjugate gradient methods in micromagnetics.
      ,
      • Kovacs A.
      • Fischbacher J.
      • Oezelt H.
      • Schrefl T.
      • Kaidatzis A.
      • Salikhov R.
      • Farle M.
      • Giannopoulos G.
      • Niarchos D.
      Micromagnetic simulations for coercivity improvement through nano-structuring of rare-earth-free L1 0-feNi magnets.
      ,
      • Exl L.
      • Fischbacher J.
      • Kovacs A.
      • Oezelt H.
      • Gusenbauer M.
      • Schrefl T.
      Preconditioned nonlinear conjugate gradient method for micromagnetic energy minimization.
      ], 3D printing [
      • Park A.
      • et al.
      Machine-Vision Assisted 3D Printing.
      ,
      • Dumas J.
      Controllable Shape Synthesis for Digital Fabrication.
      ], computer graphics [
      • Kjolstad F.
      • Kamil S.
      • Ragan-Kelley J.
      • Levin D.I.
      • Sueda S.
      • Chen D.
      • Vouga E.
      • Kaufman D.M.
      • Kanwar G.
      • Matusik W.
      • et al.
      Simit: A language for physical simulation.
      ,
      • Germer T.
      • Uelwer T.
      • Conrad S.
      • Harmeling S.
      Pymatting: A python library for alpha matting.
      ], contact mechanics [
      • Li M.
      • Ferguson Z.
      • Schneider T.
      • Langlois T.
      • Zorin D.
      • Panozzo D.
      • Jiang C.
      • Kaufman D.M.
      Incremental potential contact: Intersection-and inversion-free, large-deformation dynamics.
      ], elastodynamics [
      • Shojaei A.
      • Mossaiby F.
      • Zaccariotto M.
      • Galvanetto U.
      The meshless finite point method for transient elastodynamic problems.
      ], semiconductor device modeling [
      • Vasilevskiy A.
      • Ivanov K.
      • Konsentsiush E.
      • Redka A.
      Software for the thermal field calculation in 3D models of semiconductor devices.
      ], and topology optimization [
      • Li Y.
      • Li X.
      • Li M.
      • Zhu Y.
      • Zhu B.
      • Jiang C.
      A hybrid Lagrangian-Eulerian method for topology optimization.
      ].

      Declaration of Competing Interest

      The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

      Acknowledgment

      The development of the AMGCL library was partially funded by the state assignment to the Joint supercomputer center of the Russian academy of sciences for scientific research .

      References

        • Brandt A.
        • McCoruick S.
        • Huge J.
        Algebraic multigrid (AMG) for sparse matrix equations.
        in: Sparsity and its Applications, Vol. 257. 1985
        • Ruge J.W.
        • Stüben K.
        Algebraic multigrid.
        in: Multigrid Methods. SIAM, 1987: 73-130
        • Trottenberg U.
        • Oosterlee C.
        • Schüller A.
        Multigrid.
        Academic Press, London2001
        • Cleary A.J.
        • Falgout R.D.
        • Henson V.E.
        • Jones J.E.
        • Manteuffel T.A.
        • McCormick S.F.
        • Miranda G.N.
        • Ruge J.W.
        Robustness and scalability of algebraic multigrid.
        SIAM J. Sci. Comput. 2000; 21: 1886-1908https://doi.org/10.1137/S1064827598339402
        • Demidov D.
        AMGCL: An efficient, flexible, and extensible algebraic multigrid implementation.
        Lobachevskii J. Math. 2019; 40: 535-546https://doi.org/10.1134/S1995080219050056
        • Stüben K.
        • Clees T.
        • Klie H.
        • Lu B.
        • Wheeler M.F.
        • et al.
        Algebraic multigrid methods (AMG) for the efficient solution of fully implicit formulations in reservoir simulation.
        in: SPE Reservoir Simulation Symposium. Society of Petroleum Engineers, 2007https://doi.org/10.2118/105832-MS
        • Saleri F.
        • Veneziani A.
        Pressure correction algebraic splitting methods for the incompressible Navier–Stokes equations.
        SIAM J. Numer. Anal. 2005; 43: 174-194https://doi.org/10.1137/S0036142903435429
        • Alexandrescu A.
        Modern C++ Design: Generic Programming and Design Patterns Applied.
        Addison-Wesley, 2001
        • Barrett R.
        • Berry M.
        • Chan T.F.
        • Demmel J.
        • Donato J.
        • Dongarra J.
        • Eijkhout V.
        • Pozo R.
        • Romine C.
        • Van der Vorst H.
        Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods.
        SIAM, 1994
        • Saad Y.
        Iterative Methods for Sparse Linear Systems.
        Siam, 2003
        • Meyers S.
        Effective C++: 55 Specific Ways to Improve your Programs and Designs.
        Pearson Education, 2005
        • Demidov D.
        • Mu L.
        • Wang B.
        Accelerating linear solvers for large-scale Stokes problems with c++ metaprogramming.
        2020 (arXiv:2006.06052)
        • Balay S.
        • Abhyankar S.
        • Adams M.F.
        • Brown J.
        • Brune P.
        • Buschelman K.
        • Dalcin L.
        • Eijkhout V.
        • Gropp W.D.
        • Kaushik D.
        • Knepley M.G.
        • McInnes L.C.
        • Rupp K.
        • Smith B.F.
        • Zampini S.
        • Zhang H.
        • Zhang H.
        PETSc Users Manual.
        Argonne National Laboratory, 2016 (URL http://www.mcs.anl.gov/petsc)
        • Heroux M.A.
        • Bartlett R.A.
        • Howle V.E.
        • Hoekstra R.J.
        • Hu J.J.
        • Kolda T.G.
        • Lehoucq R.B.
        • Long K.R.
        • Pawlowski R.P.
        • Phipps E.T.
        • et al.
        An overview of the Trilinos project.
        ACM Trans. Math. Softw. 2005; 31: 397-423https://doi.org/10.1145/1089014.1089021
        • Dalton S.
        • Bell N.
        • Olson L.
        • Garland M.
        Cusp: Generic parallel algorithms for sparse matrix and graph computations.
        2014 (http://cusplibrary.github.io, Version 0.5.0)
        • Schenk O.
        • Gärtner K.
        • Fichtner W.
        • Stricker A.
        PARDISO: a high-performance serial and parallel sparse linear solver in semiconductor device simulation.
        Future Gener. Comput. Syst. 2001; 18: 69-78https://doi.org/10.1016/S0167-739X(00)00076-5
        • Dadvand P.
        • Rossi R.
        • Oñate E.
        An object-oriented environment for developing finite element codes for multi-disciplinary applications.
        Arch. Comput. Methods Eng. 2010; 17: 253-297https://doi.org/10.1007/s11831-010-9045-2
        • Lie K.-A.
        An Introduction to Reservoir Simulation using MATLAB/GNU Octave: User Guide for the MATLAB Reservoir Simulation Toolbox (MRST).
        Cambridge University Press, 2019https://doi.org/10.1017/9781108591416
        • Shangaraeva A.
        • Shevchenko D.
        Speed up of the oil saturation numerical algorithm for the plane-parallel filtration.
        Appl. Math. Sci. 2015; 9: 7467-7474https://doi.org/10.12988/ams.2015.510683
        • Goncharova G.
        • Khramchenkov M.
        Mathematical model of hydraulic fracturing of a bed.
        J. Eng. Phys. Thermophys. 2016; 89: 848-853https://doi.org/10.1007/s10891-016-1445-1
        • Cotela Dalmau J.
        • Rossi R.
        • Larese De Tetto A.
        Simulation of two-and three-dimensional viscoplastic flows using adaptive mesh refinement.
        Internat. J. Numer. Methods Engrg. 2017; https://doi.org/10.1002/nme.5574
        • Longva A.B.
        Finite Element Solutions to the Wave Equation in Non-Convex Domains - A Relaxation of the CFL Condition in the Presence of Local Mesh Refinement.
        (Master’s thesis) NTNU, 2017 (URL http://hdl.handle.net/11250/2449186)
        • Khramchenkov E.
        • Khramchenkov M.
        Numerical model of two-phase flow in dissolvable porous media and simulation of reservoir acidizing.
        Nat. Resour. Res. 2018; 27: 531-537https://doi.org/10.1007/s11053-018-9371-x
        • Krogstad S.
        • Nilsen H.
        • Møyner O.
        • Rasmussen A.
        Well control optimization of the OLYMPUS case using MRST and OPM.
        in: EAGE/TNO Workshop on OLYMPUS Field Development Optimization. European Association of Geoscientists & Engineers, 2018: cp-562https://doi.org/10.3997/2214-4609.201802287
        • Møyner O.
        • Tchelepi H.A.
        • et al.
        A mass-conservative sequential implicit multiscale method for isothermal equation-of-state compositional problems.
        SPE J. 2018; 23: 2-376https://doi.org/10.2118/182679-PA
        • Klemetsdal Ø.S.
        • Møyner O.
        • Lie K.-A.
        • et al.
        Robust nonlinear newton solver with adaptive interface-localized trust regions.
        SPE J. 2019; 24: 1-576https://doi.org/10.2118/195682-PA
        • Khramchenkov M.
        • Khramchenkov E.
        • Usmanov R.
        Non-linear equations of mechanics of swelling and metamorphic processes.
        Lobachevskii J. Math. 2019; 40: 2077-2083https://doi.org/10.1134/S1995080219120072
        • Zakirov T.
        • Galeev A.
        Absolute permeability calculations in micro-computed tomography models of sandstones by Navier-Stokes and lattice Boltzmann equations.
        Int. J. Heat Mass Transfer. 2019; 129: 415-426https://doi.org/10.1016/j.ijheatmasstransfer.2018.09.119
        • Demidov D.
        • Rossi R.
        Subdomain deflation combined with local AMG: A case study using AMGCL library.
        Lobachevskii J. Math. 2020; 41: 491-511https://doi.org/10.1134/S1995080220040071
        • Hashemi M.R.
        • Ryzhakov P.B.
        • Rossi R.
        An enriched finite element/level-set method for simulating two-phase incompressible fluid flows with surface tension.
        Comput. Methods Appl. Mech. Engrg. 2020; 370113277https://doi.org/10.1016/j.cma.2020.113277
        • Klemetsdal Ø.
        • Moncorgé A.
        • Moyner O.
        • Lie K.
        Additive Schwarz preconditioned exact Newton method as a nonlinear preconditioner for multiphase porous media flow.
        in: ECMOR XVII, no. 1. European Association of Geoscientists & Engineers, 2020: 1-20https://doi.org/10.3997/2214-4609.202035050
        • Li D.
        • Xu K.
        • Harris J.M.
        • Darve E.
        Coupled time-lapse full-waveform inversion for subsurface flow problems using intrusive automatic differentiation.
        Water Resour. Res. 2020; 56 (e2019WR027032)https://doi.org/10.1029/2019WR027032
        • Nilsen H.
        • Moncorge A.
        • Bao K.
        • Møyner O.
        • Lie K.
        • Brodtkorb A.
        Comparison between algebraic multigrid and multilevel multiscale methods for reservoir simulation.
        in: ECMOR XVII, Vol. 2020. European Association of Geoscientists & Engineers, 2020: 1-17https://doi.org/10.3997/2214-4609.202035063
        • Pinzinger R.
        • Blankenburg R.
        Speeding up the computation of the transient Richards’ equation with AMGCL.
        Water. 2020; 12: 286https://doi.org/10.3390/w12010286
        • Rasmussen A.F.
        • Sandve T.H.
        • Bao K.
        • Lauser A.
        • Hove J.
        • Skaflestad B.
        • Klöfkorn R.
        • Blatt M.
        • Rustad A.B.
        • Sævareid O.
        • Lie K.A.
        • Thune A.
        The open porous media flow reservoir simulator.
        Comput. Math. Appl. 2020; https://doi.org/10.1016/j.camwa.2020.05.014
        • Sbai M.A.
        • Larabi A.
        On solving groundwater flow and transport models with algebraic multigrid preconditioning.
        Groundwater. 2020; https://doi.org/10.1111/gwat.13016
        • Yakirevich A.
        Water flow, solute and heat transfer in groundwater.
        Water. 2020; 12https://doi.org/10.3390/w12071851
        • Isaev S.
        • Leontiev A.
        • Chudnovsky Y.
        • Nikushchenko D.
        • Popov I.
        • Sudakov A.
        Simulation of vortex heat transfer enhancement in the turbulent water flow in the narrow plane-parallel channel with an inclined oval-trench dimple of fixed depth and spot area.
        Energies. 2019; 12: 1296https://doi.org/10.3390/en12071296
        • Isaev S.
        Thermal-hydrodynamic design of energy-efficient surfaces with inclined oval-trench vortex generators.
        in: Journal of Physics: Conference Series, Vol. 1359. IOP Publishing, 2019012002https://doi.org/10.1088/1742-6596/1359/1/012002
        • Isaev S.
        • Gritckevich M.
        • Leontiev A.
        • Milman O.
        • Nikushchenko D.
        NT vortex enhancement of heat transfer and flow in the narrow channel with a dense packing of inclined one-row oval-trench dimples.
        Int. J. Heat Mass Transfer. 2019; 145118737https://doi.org/10.1016/j.ijheatmasstransfer.2019.118737
        • Isaev S.
        • Leontiev A.
        • Milman O.
        • Popov I.
        • Sudakov A.
        Influence of the depth of single-row oval-trench dimples inclined to laminar air flow on heat transfer enhancement in a narrow micro-channel.
        Int. J. Heat Mass Transfer. 2019; 134: 338-358https://doi.org/10.1016/j.ijheatmasstransfer.2018.12.175
        • Ryzhakov P.B.
        • et al.
        On the relevance of accounting for uid-structure interaction e ects in the numerical studies of type b aortic dissection.
        Biomed. J. Sci. Tech. Res. 2019; 16: 12039-12042https://doi.org/10.26717/BJSTR.2019.16.002852
      1. J. Al-Salami, C. Hu, M.M. Kamra, K. Hanada, Magnetic induction and electric potential smoothed particle magnetohydrodynamics for incompressible flows, Internat. J. Numer. Methods Fluids n/a (n/a). http://dx.doi.org/10.1002/fld.4906.

        • Ando R.
        • Batty C.
        A practical octree liquid simulator with adaptive surface resolution.
        ACM Trans. Graph. 2020; 39https://doi.org/10.1145/3386569.3392460
        • Fang Y.
        • Qu Z.
        • Li M.
        • Zhang X.
        • Zhu Y.
        • Aanjaneya M.
        • Jiang C.
        IQ-MPM: an interface quadrature material point method for non-sticky strongly two-way coupled nonlinear solids and fluids.
        ACM Trans. Graph. 2020; 39: 51https://doi.org/10.1145/3386569.3392438
        • Gagniere S.
        • Hyde D.
        • Marquez-Razon A.
        • Jiang C.
        • Ge Z.
        • Han X.
        • Guo Q.
        • Teran J.
        A hybrid Lagrangian/Eulerian collocated velocity advection and projection method for fluid simulation.
        in: Bender J. Popa T. ACM SIGGRAPH / Eurographics Symposium on Computer Animation 2020, Vol. 39. 2020
        • Isaev S.
        • Schelchkov A.
        • Leontiev A.
        • Gortyshov Y.F.
        • Baranov P.
        • Popov I.
        Vortex heat transfer enhancement in the narrow plane-parallel channel with the oval-trench dimple of fixed depth and spot area.
        Int. J. Heat Mass Transfer. 2017; 109: 40-62https://doi.org/10.1016/j.ijheatmasstransfer.2017.01.103
        • Isaev S.
        • Baranov P.
        • Popov I.
        • Sudakov A.
        • Usachov A.
        • Guvernyuk S.
        • Sinyavin A.
        • Chylunin A.
        • Mazo A.
        • Kalinin E.
        Ensuring safe descend of reusable rocket stages – Numerical simulation and experiments on subsonic turbulent air flow around a semi-circular cylinder at zero angle of attack and moderate Reynolds number.
        Acta Astronaut. 2018; 150: 117-136https://doi.org/10.1016/j.actaastro.2017.10.028
        • Isaev S.
        • Leont’ev A.
        • Mil’man O.
        • Sudakov A.
        • Usachov A.
        • Gul’tsova M.
        Intensification of heat exchange in laminar vortex air flow in a narrow channel with a row of inclined oval trenches.
        J. Eng. Phys. Thermophys. 2018; 91: 963-974https://doi.org/10.1007/s10891-018-1822-z
        • Isaev S.
        • Baranov P.
        • Popov I.
        • Sudakov A.
        • Usachov A.
        • Guvernyuk S.
        • Sinyavin A.
        • Chulyunin A.
        • Mazo A.
        • Demidov D.
        • et al.
        Numerical simulation and experiments on turbulent air flow around the semi-circular profile at zero angle of attack and moderate Reynolds number.
        Comput. & Fluids. 2019; 188: 1-17https://doi.org/10.1016/j.compfluid.2019.03.013
        • Isaev S.
        • Leontiev A.
        • Milman O.
        • Nikushchenko D.
        • Sudakov A.
        The effect of anomalous enhancement of vortex heat transfer in the separated flow in inclined oval-trench dimple and on the structured surfaces.
        in: AIP Conference Proceedings, Vol. 2211. AIP Publishing LLC, 2020020003https://doi.org/10.1063/5.0000539
        • Isaev S.
        • Leontiev A.
        • Milman O.
        • Nikushchenko D.
        • Egorova A.
        Energy-efficient surface of air capacitors with inclined single-row oval-trench dimples and protrusions.
        J. Phys. Conf. Ser. 2020; 1565012001https://doi.org/10.1088/1742-6596/1565/1/012001
        • Parag T.
        • Plaza S.
        • Scheffer L.
        Small sample learning of superpixel classifiers for EM segmentation.
        in: Golland P. Hata N. Barillot C. Hornegger J. Howe R. Medical Image Computing and Computer-Assisted Intervention – MICCAI 2014. Springer International Publishing, Cham2014: 389-397https://doi.org/10.1007/978-3-319-10404-1˙49
        • Xu K.
        • Tartakovsky A.M.
        • Burghardt J.
        • Darve E.
        Inverse modeling of viscoelasticity materials using physics constrained learning.
        2020 (arXiv:2005.04384)
        • Xu K.
        • Darve E.
        Physics constrained learning for data-driven inverse modeling from sparse observations.
        2020 (arXiv:2002.10521)
        • Fischbacher J.
        • Kovacs A.
        • Oezelt H.
        • Schrefl T.
        • Exl L.
        • Fidler J.
        • Suess D.
        • Sakuma N.
        • Yano M.
        • Kato A.
        • et al.
        Nonlinear conjugate gradient methods in micromagnetics.
        AIP Adv. 2017; 7045310https://doi.org/10.1063/1.4981902
        • Kovacs A.
        • Fischbacher J.
        • Oezelt H.
        • Schrefl T.
        • Kaidatzis A.
        • Salikhov R.
        • Farle M.
        • Giannopoulos G.
        • Niarchos D.
        Micromagnetic simulations for coercivity improvement through nano-structuring of rare-earth-free L1 0-feNi magnets.
        IEEE Trans. Magn. 2017; 53: 1-5https://doi.org/10.1109/TMAG.2017.2701418
        • Exl L.
        • Fischbacher J.
        • Kovacs A.
        • Oezelt H.
        • Gusenbauer M.
        • Schrefl T.
        Preconditioned nonlinear conjugate gradient method for micromagnetic energy minimization.
        Comput. Phys. Comm. 2019; 235: 179-186https://doi.org/10.1016/j.cpc.2018.09.004
        • Park A.
        • et al.
        Machine-Vision Assisted 3D Printing.
        (Ph.D. thesis) Massachusetts Institute of Technology, 2016 (URL http://hdl.handle.net/1721.1/113162)
        • Dumas J.
        Controllable Shape Synthesis for Digital Fabrication.
        (Theses) Université de Lorraine, 2017 (URL https://tel.archives-ouvertes.fr/tel-01547313)
        • Kjolstad F.
        • Kamil S.
        • Ragan-Kelley J.
        • Levin D.I.
        • Sueda S.
        • Chen D.
        • Vouga E.
        • Kaufman D.M.
        • Kanwar G.
        • Matusik W.
        • et al.
        Simit: A language for physical simulation.
        ACM Trans. Graph. 2016; 35: 1-21https://doi.org/10.1145/2866569
        • Germer T.
        • Uelwer T.
        • Conrad S.
        • Harmeling S.
        Pymatting: A python library for alpha matting.
        J. Open Source Softw. 2020; 5: 2481https://doi.org/10.21105/joss.02481
        • Li M.
        • Ferguson Z.
        • Schneider T.
        • Langlois T.
        • Zorin D.
        • Panozzo D.
        • Jiang C.
        • Kaufman D.M.
        Incremental potential contact: Intersection-and inversion-free, large-deformation dynamics.
        ACM Trans. Graph. 2020; 39https://doi.org/10.1145/3386569.3392425
        • Shojaei A.
        • Mossaiby F.
        • Zaccariotto M.
        • Galvanetto U.
        The meshless finite point method for transient elastodynamic problems.
        Acta Mech. 2017; 228: 3581-3593https://doi.org/10.1007/s00707-017-1894-4
        • Vasilevskiy A.
        • Ivanov K.
        • Konsentsiush E.
        • Redka A.
        Software for the thermal field calculation in 3D models of semiconductor devices.
        Electron. Eng. Ser. 2. Semicond. Devices. 2020; 1: 49-56https://doi.org/10.36845/2073-8250-2020-256-1-49-56
        • Li Y.
        • Li X.
        • Li M.
        • Zhu Y.
        • Zhu B.
        • Jiang C.
        A hybrid Lagrangian-Eulerian method for topology optimization.
        2020 (arXiv:2003.01215)