Research and development of fluid analysis tools using GPU
JAXA Supercomputer System Annual Report February 2022-January 2023
Report Number: R22EDA201G24
Subject Category: Aeronautical Technology
- Responsible Representative: Takashi Aoyama, Aviation Program Director, Aviation Technology
- Contact Information: Andrea Sansica(sansica.andrea@jaxa.jp)
- Members: David Lusher, Andrea Sansica, Paul Zehner
Abstract
Evaluation of FaSTAR and OpenSBLI acceleration using GPU for buffet analysis.
Reference URL
Please refer to ‘数値解析技術の研究 | 基盤技術の研究(fundamental research) | JAXA航空技術部門‘.
Reasons and benefits of using JAXA Supercomputer System
To develop a GPU version of CFD solvers, the GPU nodes available on JSS3 can be used to verify the code and run large scale simulations
Achievements of the Year
About FaSTAR-GPU: Acceleration and optimizion of FaSTAR on GPU using OpenACC. Asynchronous execution of logging kernels in the background (figure 1): this allows a speedup of 15-21% for a memory overhead of 4-8%. Multi-GPU version of the code (figure 2): FaSTAR-GPU has strong scaling similar to other multi-GPU solvers and a good weak scaling.
About OpenSBLI: OpenSBLI has been developed on JSS TOKI-RURI GPU nodes to perform multi-block, high-fidelity, high-speed buffet analysis. The validation of OpenSBLI was done and high-speed 3D buffett Direct Numerical Simulations could be carried out (video 1). High-fidelity channel flow simulations were also performed to improve turbulent Prantdl number scaling used in RANS calculations (figure 3).

Fig.2: Strong and weak scaling efficiency of multi-GPU execution, based on NASA CRM case (3 million to 46 million cells).

Fig.3: High fidelity channel flow: Investigation of turbulent Prandtl number dependence on thermal wall boundary condition.
Fig.4(video1): Transonic Buffett OpenSBLI simulation for NASA-CRM wing on JSS3 TOKI-RURI GPU nodes.
Publications
– Peer-reviewed papers
[1] P. Zehner and A. Hashimoto, Acceleration of the data-parallel lower-upper relaxation time-integration method on GPU for an unstructured CFD solver, Computers & Fluids, 2023.
[2] D.J. Lusher, G.N. Coleman. Numerical study of compressible wall-bounded turbulence – the effect of thermal wall conditions on the turbulent Prandtl number in the low-supersonic regime. International Journal of Computational Fluid Dynamics, 2023.
– Non peer-reviewed papers
[1] P. Zehner and A. Hashimoto, Asynchronous Execution of Logging Kernels in a GPU Accelerated CFD Solver, in Proceedings of the 54th Fluid Dynamics Conference / the 40th Aerospace Numerical Simulation Symposium, Morioka, Japan, Jun. 2022, vol. JAXA-SP-22-007, pp. 331–339. [Online]. Available: http://id.nii.ac.jp/1696/00049141/
[2] D.J. Lusher, M. Zauner, A. Sansica, A. Hashimoto. Automatic Code-Generation to Enable High-Fidelity Simulations of Multi-Block Airfoils on GPUs. AIAA SciTech (2023) conference proceedings.
– Oral Presentations
[1] P. Zehner and A. Hashimoto, Asynchronous Execution of Logging Kernels in a GPU Accelerated CFD Solver, in Proceedings of the 54th Fluid Dynamics Conference / the 40th Aerospace Numerical Simulation Symposium, Morioka, Japan, 2022
[2] P. Zehner and A. Hashimoto, Influence of Time Integration Method on GPU Performance for Industry Relevant CFD Simulations, Candar 2022 conference workshop, GCA’22, Himeji, Japan, Nov. 2022.
[3] D.J. Lusher, A. Sansica, A. Hashimoto. Towards High-Fidelity Transonic Buffet Simulations By Using An Automatic CFD Code-Generation System On Heterogeneous Exa-scale Supercomputers. ANSS Conference, 2022.
[4] D.J. Lusher, M. Zauner, A. Sansica, A. Hashimoto. Automatic Code-Generation to Enable High-Fidelity Simulations of Multi-Block Airfoils on GPUs. AIAA SciTech conference, 2023.
– Poster Presentations
[1]D.J. Lusher, A. Sansica, A. Hashimoto. OpenSBLI: Automated code-generation for high-fidelity airfoil simulations on heterogeneous HPC architectures. SuperComputing 22 conference, Dallas (2022).
Usage of JSS
Computational Information
- Process Parallelization Methods: GPU
- Thread Parallelization Methods: N/A
- Number of Processes: 128
- Elapsed Time per Case: 100 Hour(s)
JSS3 Resources Used
Fraction of Usage in Total Resources*1(%): 0.68
Details
Please refer to System Configuration of JSS3 for the system configuration and major specifications of JSS3.
System Name | CPU Resources Used(Core x Hours) | Fraction of Usage*2(%) |
---|---|---|
TOKI-SORA | 126682.01 | 0.01 |
TOKI-ST | 290.13 | 0.00 |
TOKI-GP | 1222048.04 | 51.98 |
TOKI-XM | 0.00 | 0.00 |
TOKI-LM | 0.00 | 0.00 |
TOKI-TST | 0.00 | 0.00 |
TOKI-TGP | 0.00 | 0.00 |
TOKI-TLM | 0.00 | 0.00 |
File System Name | Storage Assigned(GiB) | Fraction of Usage*2(%) |
---|---|---|
/home | 128.67 | 0.12 |
/data and /data2 | 20500.00 | 0.16 |
/ssd | 3433.33 | 0.48 |
Archiver Name | Storage Used(TiB) | Fraction of Usage*2(%) |
---|---|---|
J-SPACE | 4.11 | 0.02 |
*1: Fraction of Usage in Total Resources: Weighted average of three resource types (Computing, File System, and Archiver).
*2: Fraction of Usage:Percentage of usage relative to each resource used in one year.
ISV Software Licenses Used
ISV Software Licenses Used(Hours) | Fraction of Usage*2(%) | |
---|---|---|
ISV Software Licenses(Total) | 16.99 | 0.01 |
*2: Fraction of Usage:Percentage of usage relative to each resource used in one year.
JAXA Supercomputer System Annual Report February 2022-January 2023