本文へ移動

サイトナビゲーションへ移動

検索ボックスへ移動

サイドバーへ移動

ここは、本文エリアの先頭です。

Research and development of fluid analysis tools using GPU

JAXA Supercomputer System Annual Report February 2022-January 2023

Report Number: R22EDA201G24

Subject Category: Aeronautical Technology

PDF available here

  • Responsible Representative: Takashi Aoyama, Aviation Program Director, Aviation Technology
  • Contact Information: Andrea Sansica(sansica.andrea@jaxa.jp)
  • Members: David Lusher, Andrea Sansica, Paul Zehner

Abstract

Evaluation of FaSTAR and OpenSBLI acceleration using GPU for buffet analysis.

Reference URL

Please refer to ‘数値解析技術の研究 | 基盤技術の研究(fundamental research) | JAXA航空技術部門‘.

Reasons and benefits of using JAXA Supercomputer System

To develop a GPU version of CFD solvers, the GPU nodes available on JSS3 can be used to verify the code and run large scale simulations

Achievements of the Year

About FaSTAR-GPU: Acceleration and optimizion of FaSTAR on GPU using OpenACC. Asynchronous execution of logging kernels in the background (figure 1): this allows a speedup of 15-21% for a memory overhead of 4-8%. Multi-GPU version of the code (figure 2): FaSTAR-GPU has strong scaling similar to other multi-GPU solvers and a good weak scaling.

About OpenSBLI: OpenSBLI has been developed on JSS TOKI-RURI GPU nodes to perform multi-block, high-fidelity, high-speed buffet analysis. The validation of OpenSBLI was done and high-speed 3D buffett Direct Numerical Simulations could be carried out (video 1). High-fidelity channel flow simulations were also performed to improve turbulent Prantdl number scaling used in RANS calculations (figure 3).

Annual Reoprt Figures for 2022

Fig.1: Timeline of kernels execution, based on NASA CRM case (11 million cells);

 

Annual Reoprt Figures for 2022

Fig.2: Strong and weak scaling efficiency of multi-GPU execution, based on NASA CRM case (3 million to 46 million cells).

 

Annual Reoprt Figures for 2022

Fig.3: High fidelity channel flow: Investigation of turbulent Prandtl number dependence on thermal wall boundary condition.

 

Fig.4(video1): Transonic Buffett OpenSBLI simulation for NASA-CRM wing on JSS3 TOKI-RURI GPU nodes.

Publications

– Peer-reviewed papers

[1] P. Zehner and A. Hashimoto, Acceleration of the data-parallel lower-upper relaxation time-integration method on GPU for an unstructured CFD solver, Computers & Fluids, 2023.

[2] D.J. Lusher, G.N. Coleman. Numerical study of compressible wall-bounded turbulence – the effect of thermal wall conditions on the turbulent Prandtl number in the low-supersonic regime. International Journal of Computational Fluid Dynamics, 2023.

– Non peer-reviewed papers

[1] P. Zehner and A. Hashimoto, Asynchronous Execution of Logging Kernels in a GPU Accelerated CFD Solver, in Proceedings of the 54th Fluid Dynamics Conference / the 40th Aerospace Numerical Simulation Symposium, Morioka, Japan, Jun. 2022, vol. JAXA-SP-22-007, pp. 331–339. [Online]. Available: http://id.nii.ac.jp/1696/00049141/

[2] D.J. Lusher, M. Zauner, A. Sansica, A. Hashimoto. Automatic Code-Generation to Enable High-Fidelity Simulations of Multi-Block Airfoils on GPUs. AIAA SciTech (2023) conference proceedings.

– Oral Presentations

[1] P. Zehner and A. Hashimoto, Asynchronous Execution of Logging Kernels in a GPU Accelerated CFD Solver, in Proceedings of the 54th Fluid Dynamics Conference / the 40th Aerospace Numerical Simulation Symposium, Morioka, Japan, 2022

[2] P. Zehner and A. Hashimoto, Influence of Time Integration Method on GPU Performance for Industry Relevant CFD Simulations, Candar 2022 conference workshop, GCA’22, Himeji, Japan, Nov. 2022.

[3] D.J. Lusher, A. Sansica, A. Hashimoto. Towards High-Fidelity Transonic Buffet Simulations By Using An Automatic CFD Code-Generation System On Heterogeneous Exa-scale Supercomputers. ANSS Conference, 2022.

[4] D.J. Lusher, M. Zauner, A. Sansica, A. Hashimoto. Automatic Code-Generation to Enable High-Fidelity Simulations of Multi-Block Airfoils on GPUs. AIAA SciTech conference, 2023.

– Poster Presentations

[1]D.J. Lusher, A. Sansica, A. Hashimoto. OpenSBLI: Automated code-generation for high-fidelity airfoil simulations on heterogeneous HPC architectures. SuperComputing 22 conference, Dallas (2022).

Usage of JSS

Computational Information

  • Process Parallelization Methods: GPU
  • Thread Parallelization Methods: N/A
  • Number of Processes: 128
  • Elapsed Time per Case: 100 Hour(s)

JSS3 Resources Used

 

Fraction of Usage in Total Resources*1(%): 0.68

 

Details

Please refer to System Configuration of JSS3 for the system configuration and major specifications of JSS3.

Computational Resources
System Name CPU Resources Used
(Core x Hours)
Fraction of Usage*2(%)
TOKI-SORA 126682.01 0.01
TOKI-ST 290.13 0.00
TOKI-GP 1222048.04 51.98
TOKI-XM 0.00 0.00
TOKI-LM 0.00 0.00
TOKI-TST 0.00 0.00
TOKI-TGP 0.00 0.00
TOKI-TLM 0.00 0.00

 

File System Resources
File System Name Storage Assigned
(GiB)
Fraction of Usage*2(%)
/home 128.67 0.12
/data and /data2 20500.00 0.16
/ssd 3433.33 0.48

 

Archiver Resources
Archiver Name Storage Used
(TiB)
Fraction of Usage*2(%)
J-SPACE 4.11 0.02

*1: Fraction of Usage in Total Resources: Weighted average of three resource types (Computing, File System, and Archiver).

*2: Fraction of Usage:Percentage of usage relative to each resource used in one year.

 

ISV Software Licenses Used

ISV Software Licenses Resources
ISV Software Licenses Used
(Hours)
Fraction of Usage*2(%)
ISV Software Licenses
(Total)
16.99 0.01

*2: Fraction of Usage:Percentage of usage relative to each resource used in one year.

JAXA Supercomputer System Annual Report February 2022-January 2023