PyFR: A GPU-Accelerated Next-Generation Computational Fluid Dynamics Python Framework

PyFR is an open-source 5,000 line Python based framework for solving fluid-flow problems that can exploit many-core computing hardware such as GPUs! Computational simulation of fluid flow, often referred to as Computational Fluid Dynamics (CFD), plays an critical role in the aerodynamic design of numerous complex systems, including aircraft, F1 racing cars, and wind turbines. CFD technology allows engineers to understand complex flow patterns, and thus perform aerodynamic design, without ever flying an aircraft, or even firing up a wind tunnel. However, current generation CFD software packages do not meet industrial needs (in terms of accuracy and efficiency). New mathematics – in particular so called ‘high-order accurate’ numerical algorithms – and new software development paradigms – that make effective use of next generation ‘many-core’ computing platforms – are required in order to advance industrial CFD capabilities to a new level of performance.

Update Sept 6, 2014: The PyFR team has published a new paper, “Heterogeneous Computing on Mixed Unstructured Grids with PyFR“, showing this Python framework can perform high-order accurate unsteady simulations of flow on mixed unstructured grids using heterogeneous multi-node hardware. Specifically, after benchmarking single-node performance for various platforms, PyFR v0.2.2 is used to undertake simulations of unsteady flow over a circular cylinder at Reynolds number 3 900 using a mixed unstructured grid of prismatic and tetrahedral elements on a desktop workstation containing an Intel Xeon E5-2697 v2 CPU, an NVIDIA Tesla K40c GPU, and an AMD FirePro W9100 GPU. Both the performance and accuracy of PyFR are assessed in the paper.

What is PyFR?

PyFR is an open-source Python based framework for solving fluid-flow problems using the Flux Reconstruction approach of Huynh. The framework offers high-order accuracy in space, even on mixed unstructured meshes around complex engineering geometries. It is also designed to target a range of hardware platforms, including clusters of CPUs, Nvidia GPUs, and AMD GPUs, via use of a light-weight in-built domain specific language derived from the Mako templating engine.

The current release (PyFR 0.2.1) has the following capabilities:

Governing equations – Euler, compressible Navier-Stokes
Dimensionality – 2D, 3D
Element types – triangles, quadrilaterals, hexahedra, prisms, tetrahedra
Platforms – CPU clusters, Nvidia GPU clusters, AMD GPU clusters
Spatial discretisation – high-order FR
Temporal discretisation – explicit Runge-Kutta schemes
Precision – single, double
Mesh files read – Gmsh (.msh)
Solution files produced – unstructured VTK (.vtu)

All this from just 5,000 lines of Python!

What is PyFR for?

The majority of current-generation industry standard CFD technology is based on a so-called Reynolds Averaged Navier-Stokes (RANS) approach, which has limited applicability. Specifically, if relevant flow phenomena are fundamentally unsteady, RANS is not fit-for-purpose. PyFR aims to expand the industrial CFD envelope from its current ‘RANS plateau’; enabling affordable and accurate simulation of currently intractable unsteady flow problems via scale resolving approaches such as Large Eddy Simulations (LES). As such, it is envisaged PyFR will have significant impact in a range of application areas including:

Design of next-generation unmanned aerial vehicles
Design of F1 racing cars
Design of jet engines
Design of wind turbines
Aircraft noise reduction

Flow over a spoiler deployed at 90 degrees to the oncoming flow, computed on a mesh with 1.3 billion degrees of freedom using 184 x Nvidia M2090 GPUs (Emerald HPC facility at the Centre for Innovation UK).

Flow over a low-pressure turbine blade, computed on a mesh with 85 million degrees of freedom using 4 x Nvidia K20 GPUs (desktop workstation).

Who is Developing PyFR?

PyFR is being developed in the Vincent Lab, Department of Aeronautics, Imperial College London.

Project Lead: Dr Peter Vincent – Senior Lecturer in Aeronautics at Imperial College London

Dr Peter Vincent

Developer: Freddie Witherden – PhD Student in Aeronautics at Imperial College London

Freddie Witherden

Developer: Antony Farrington – PhD Student in Aeronautics at Imperial College London

Antony Farrington

Developer: George Ntemos – PhD Student in Aeronautics at Imperial College London

George Ntemos

Developer: Harry Davis – PhD Student in Aeronautics at Imperial College London

Harry Davis

Developer: Brian Vermeire – Postdoctoral Research Associate in Aeronautics at Imperial College London

Brian Vermeire

Can I use PyFR?

Yes! PyFR is available for free under an industry-friendly open-source New Style BSD License. You can download it from our website (www.pyfr.org), or fork our PyFR repo from Github (https://github.com/vincentlab/PyFR). PyFR is also available in the cloud via Rescale (www.rescale.com).

Funding

Development of PyFR is directly supported by a £1.3M EPSRC Early Career Fellowship held by Peter Vincent (EP/K027379/1), as well as three EPSRC PhD studentships, one Airbus/EPSRC PhD studentship, and hardware donations from Nvidia, AMD, and Intel. The research is also partially supported by a £1.6M EPSRC Platform Grant (EP/L000407/1), and a £0.8M EPSRC UK Turbulence Consortium Grant (EP/L000261/1). Finally, the PyFR team are grateful for access to the Emerald HPC facility provided by the Centre for Innovation.