JAXA Supercomputer System Annual Report February 2025-January 2026

Report Number: R25EG3212

Subject Category: Research and Development

PDF (to be added)

• Responsible Representative: Taro Shimizu, Director, Research and Development Directorate, Research Unit III
• Contact Information: Takanori Haga, Research and Development Directorate, Research Unit III(haga.takanori@jaxa.jp)
• Members: Junya Aono, Masaharu Abe, Yu Daimon, Yuma Fukushima, Takanori Haga, Morimasa Hattori, Shotaro Hamato, Hiroyuki Ito, Ryohei Kirihara, Hideyo Negishi, Takenori Nakajima, Shinji Ohno, Yasuhito Okano, Taro Shimizu, Seiji Tsutsumi, Ryoji Takaki, Osamu Watanabe, Himeko Yamamoto, Patrick Strempfl

Abstract

In order to capture the unsteady phenomenon in a real-scale liquid rocket engine, the relevant physical models and numerical methods necessary for combustion LES are developed. An analysis tool is validated for the subscale test data, and applied to the development of a real-scale engine.

Reference URL

Please refer to https://stage.tksc.jaxa.jp/jedi/en/simul/index.html .

Reasons and benefits of using JAXA Supercomputer System

Since the flow and combustion in rocket chambers are in a turbulent state and have nonstationary characteristics, LES analysis is essential. Even in this verification target, analysis calculation of about several million steps is required for grid of tens to- hundreds of millions of cells, so it is impossible to achieve the target without using supercomputer.

Achievements of the Year

The results of this year are as follows.

1. Development of Robust Wall-Modeled LES for Reacting Flows over Extremely Cold Walls

We developed a robust wall-modeled large-eddy simulation (WMLES) framework to accurately predict wall heat flux, which is essential for the cooling design of liquid rocket engine combustors. While Reynolds averaged Navier-Stokes simulations are computationally inexpensive, they often struggle to capture unsteady phenomena such as separation and recirculation. In contrast, LES can predict such unsteady features with high fidelity, but its computational cost becomes prohibitive under practical engine conditions. To address this issue, we applied WMLES to combustor flows. However, the combination of the Flux Reconstruction (FR) method and the wall model is prone to numerical instability around the near wall region especially under cold wall conditions. The WMLES often diverges due the numerical instability. To enhance numerical robustness, we introduced artificial viscosity only in the near wall region. For the reacting turbulent channel flow, the predicted boundary layer profiles and wall heat flux agree well with those obtained from wall resolved LES. Furthermore, for the single-element rocket combustor experiment conducted at Technical University of Munich, LES without the wall model underestimates the wall heat flux, whereas the WMLES shows good agreement with the experimental data (Fig. 1). This achievement has established the basis of the numerical method for quantitatively assessing the cooling design of rocket engine combustors. The total number of computational points was approximately 74 million, and the computation time was 200 hours (using 250 Fujitsu A64FX CPUs, 50,000 NH).

2. Accelerating the Performance of LS-FLOW-HO on GPUs Using OpenACC

GPU-based systems are becoming mainstream in HPC environments, while CPU-based supercomputers are becoming increasingly insufficient for rocket engine combustion simulations. To address this issue, we are working to enhance the performance of LS-FLOW-HO by porting it to GPUs. This year, we implemented a GPU-enabled version of LS-FLOW-HO using OpenACC, enabling LES simulations of liquid rocket engine combustors on NVIDIA V100 GPUs. In addition to applying OpenACC, further optimizations such as loop collapsing and improving memory access continuity achieved a 2.6x speedup, exceeding the theoretical CPU-to-GPU performance ratio of 2.3x. Furthermore, in a practical-scale case involving a 20‑million‑point single-element combustor LES, we computed 0.751 msec of physical simulation time using 64 GPUs in 2.25 hours, demonstrating that the achieved performance is suitable for real-world use (Fig. 2).

3. Verification and validation of LS-FLOW-HO for gaseous reacting flows

First, the effect of the positivity preserving and entropy limiter is investigated for different bounds on density variation for different order of the FR scheme (P1 and P2) on a GOX/GH2 2D shear layer flame (Fig. 3). A mesh dependency study revealed the necessity of resolving the post tip with at least 30 points to capture fuel entrainment into the post tip recirculation. P2 simulations could successfully be performed, which is important for the application of a wall model. In context of the limiter it was shown that additional bounds on density support the convergence of P2, revealing a trade-off : loose bounds on density result in improved temperature distribution, however in can lead to instabilities progressing downstream.

Second, a validation test of the PennState preburner PSU-RCM1 test bench was conducted to investigate LS-FLOW-HO's performance on a real case focusing on acoustics and heat flux prediction. A fully 3D compressible reacting flow simulation is performed. The simulation of the PSU-RMC1 is successfully performed and two acoustic modes could be captured: a longitudinal mode at 2500 Hz, corresponding to the first longitudinal mode and an additional 1R mode at 50 kHz. The obtained acoustic modes are in good agreement with other numerical studies found in literature. The prediction of the heat flux however requires additional investigation since the current results show a significantly underpredicted heat flux compared to the experimental data. The total number of computational points was approximately 24 million, and the computation time was 230 hours (using 256 CPUs, 58,880 NH).

4. Development of a Gas-Liquid Two-Phase Solver Based on a Diffused Interface Model

Since LS-FLOW-HO employs a single-phase fluid model, its scope of application has been limited to supercritical pressure combustion. However, depending on operating conditions—such as throttling in reusable engines or upper-stage engines—combustion may occur at subcritical pressures; therefore, we are working to introduce a two-phase fluid model. As an algorithm suitable for large-scale parallel computing on GPUs, we conducted a validation analysis of a fluid solver based on an diffused interface model. We performed a three-dimensional unsteady analysis of the coaxial injection of liquid nitrogen and gaseous nitrogen under subcritical pressure. We analyzed two cases—with and without an evaporation model—and compared the flow fields (Fig. 4). The simulation used approximately 14 million grid points and took 48 hours to complete (using 100 CPUs, 4,800 NH).

Fig.1: Temperature distribution and wall heat flux profile in TUM single-element combustor.

Fig.2: Speedup of each routine: Comparison between GPU (NVIDIA V100) and CPU (Fujitsu FX1000).

Fig.3: Instantaneous temperature field of the 2D shear layer field

Fig.4: Distribution of LN₂ in subcritical-pressure LN₂/GN₂ coaxial injection (t=29.4 ms, 1 million steps)

Publications

- Peer-reviewed papers

1) Okano, Y., Haga, T., "Robust Wall-Modeled Large Eddy Simulation with Flux Reconstruction Scheme under Extremely Cold Wall Conditions," Computers & Fluids, Volume 314, 107109

- Non peer-reviewed papers

1) Watanabe, O., Haga, T., Takaki, R., "Towards GPU Implementation of a high-order combustion solver LS-FLOW-HO in the Post-Fugaku Era," 57th FDC and 43nd ANSS. (in Japanese)

2) Okano, Y., Haga, T., "Robust Isothermal-Wall WMLES with the Flux Reconstruction Method," 57th Fluid Dynamics Conference / 43rd Aerospace Numerical Simulation Technology Symposium (in Japanese).

3) Okano, Y., Haga, T., "Robust Wall-modeled LES with Flux Reconstruction Scheme toward Accurate Heat Flux Prediction in Rocket Engine Combustors," AIAA Paper, No. 2026-0387, 2026.

4) Haga, T., "Toward resolution improvement of turbulent diffusion flames at supercritical pressure by high-order flux-reconstruction method," 57th FDC and 43nd ANSS. (in Japanese)

- Invited Presentations

1) Haga, T., "Towards Prediction of Combustion Instability in Liquid Rocket Engine Combustors by High-fidelity Simulation," SoTiC 2025: Symposium on Thermoacoustics in Combustion: Industry meets Academia.

- Oral Presentations

1) Watanabe, O., Haga, T., Takaki, R., "GPU Acceleration and Performance Optimization of a High-Order Combustion Solver LS-FLOW-HO Using OpenACC," 39th CFD symposium. (in Japanese)

2) Watanabe, O., Haga, T., Takaki, R., "GPU Acceleration and Performance Optimization of a High-Order Combustion Solver LS-FLOW-HO Using OpenACC," OpenACC User Workshop at SCA2026.

3) Okano, Y., Haga, T., "Wall-Modeled LES of Reacting Flows under Cold-Wall Conditions Using the Flux Reconstruction Method," 39th Computational Fluid Dynamics Symposium, December 16, 2025, Kitakyushu, Japan (in Japanese).

4) Strempfl, P. and Haga,T., "LES of a GH2/GO2 shear layer flame under rocket engine like conditions using Flux Reconstruction" 39th CFD Symposium (in English)

5) Haga, T., "Extension of a high-order modal limiter for supercritical diffusion flame flows," 39th CFD symposium. (in Japanese)

6) Haga, T., "Case Study of In-Situ/In-Transit Visualization Using JAXA's In-House CFD Solvers," HPDMV'26 at SCA2026.

Usage of JSS

Computational Information

• Process Parallelization Methods: MPI
• Thread Parallelization Methods: OpenMP
• Number of Processes: 1 - 20000
• Elapsed Time per Case: 240 Hour(s)

JSS3 Resources Used

Fraction of Usage in Total Resources*1(%): 4.11

Details

Please refer to System Configuration of JSS3 for the system configuration and major specifications of JSS3.

*1: Fraction of Usage in Total Resources: Weighted average of three resource types (Computing, File System, and Archiver).

*2: Fraction of Usage：Percentage of usage relative to each resource used in one year.

ISV Software Licenses Used

*2: Fraction of Usage：Percentage of usage relative to each resource used in one year.

JAXA Supercomputer System Annual Report February 2025-January 2026