JAXA Supercomputer System Annual Report April 2020-March 2021

Report Number: R20EG3212

Subject Category: Research and Development

PDF available here

• Responsible Representative: Taro Shimizu, Director, Research and Development Directorate, Research Unit III
• Contact Information: Takanori Haga, Research and Development Directorate, Research Unit III(haga.takanori@jaxa.jp)
• Members: Takayuki Ito, Ryoji Takaki, Seiji Tsutsumi, Hiroyuki Ito, Taro Shimizu, Junya Aono, Takanori Haga, Masaharu Abe, Masayuki Kakehi, Mikiroh Motoe, Manabu Hisida, Hideyo Negishi, Yoichi Ohnishi, Miki Nishimoto, Yu Daimon, Ashvin Hosangadi, Osamu Fukasawa, Shinji Ohno, Andrea Zambon, Takenori Nakajima, Takashi Amemiya, Yutaka Umemura, Hironori Fujiwara, Hiroumi Tani, Keiichiro Fujimoto, Tetsufumi Ohmaru, Taroh Fukuda, Masashi Toyama, Kei Nishimura, Daiki Muto, Nozomu Kanno, Takeshi Watanabe, Yuma Fukushima, Kazuma Tago, Kiyoshi Kumahata

Abstract

In order to capture the unsteady phenomenon in a real-scale liquid rocket engine, the relevant physical models and numerical methods necessary for combustion LES are developed. An analysis tool is validated for the subscale test data, and applied to the development of a real-scale engine.

Reference URL

Please refer to https://stage.tksc.jaxa.jp/jedi/en/simul/index.html .

Reasons and benefits of using JAXA Supercomputer System

Since the flow and combustion in rocket chambers are in a turbulent state and have nonstationary characteristics, LES analysis is essential. Even in this verification target, analysis calculation of about several million steps is required for grid of tens of millions of cells, so it is impossible to achieve the target without using supercomputer.

Achievements of the Year

Towards an unsteady simulation of a full-scale rocket combustion chamber, large eddy simulation (LES) of a methane/oxygen subscale combustion chamber is performed. A JAXA's in-house CFD solver LS-FLOW is used in the simulation. A new wall model for hydrocarbon reacting flows is applied to characterize the near-wall flow. For combustion modeling, the non-adiabatic flamelet progress variable approach is used. The number of grid point is approximately 42 million. Figure 1 shows the instantaneous temperature field. The present simulation well captures the unsteady mixing structure, the complex turbulent combustion field, and the development of the boundary layer.

Aiming at full-scale combustor simulation by improving the resolution and reducing the computational cost of LES, we have developed an in-house CFD solver LS-FLOW-HO based on the flux reconstruction method, which is a high-order unstructured grid method. This year, we developed a wall model of the turbulent boundary layer necessary for predicting the wall heat flux of a full-scale combustor, and succeeded in introducing real gas models that are indispensable for analysis of high-pressure combustors.

First, we developed a wall model LES methodology that combines a wall stress model for non-reacting flows and a flux reconstruction method. In particular, we investigated the effect of wall boundary conditions (BC) when the flow field near the wall is not sufficiently resolved by the grid, and developed a stable and highly accurate BC treatment method. Flow over periodic hills with detachment and reattachment is computed as a verification. In the periodic hills flow, the computation becomes unstable in the large recirculation region, however it is confirmed that the computation becomes stable and accurate with the developed treatment in this study (Figure 2). Currently, a validation study of the wall model extended to reacting flows is underway, and the developed wall-modeled LES is expected to improve the prediction of wall heat transfer in the full-scale combustion chamber.

Next, the SRK equation of state and Chung's transport coefficients model were adopted to calculate the mixing of cryogenic liquid oxygen and gas hydrogen injected into the rocket engine combustor. In order to stably calculate the interface with a very large density ratio, we introduced a limiter that maintains the positivity of the approximate polynomials. The flamelet progress variable method was adopted for the combustion model. The validation case is a coaxial LOX / H2 single injector, which corresponds to the experiment of 6 MPa conducted at the P8 test facility of DLR Lampoldshausen. The number of computational cells (hexa, second-order element) is about 860,000, and the degree of freedom is about 23 million points (p2, third order accuracy). Figure 3 shows the computational result of the temperature distribution. Using 320 nodes (1280 processes x 12 threads, MPI / OpenMP hybrid parallelization) of the JSS3 TOKI-SORA (FX 1000) system, the wall clock time required for analysis with a physical time of 10 ms was about 27 hours.

Fig.1: Instantaneous temperature field of a methane/oxygen subscale combustion chamber.

Fig.2: Iso-surface of the second invariant of velocity gradient tensor for the periodic hill channel flow (colored by mainstream velocity).

Fig.3(video): Turbulent combustion LES for the LOX / H2 coaxial single injector (temperature distribution, movie).

Publications

- Peer-reviewed papers

1) Daiki Muto, Yu Daimon, Hideyo Negishi, Taro Shimizu, Wall modeling of turbulent methane/oxygen reacting flows for predicting heat transfer, International Journal of Heat and Fluid Flow, 87, 108755, 2021, 10.1016/j.ijheatfluidflow.2020.108755.

- Non peer-reviewed papers

1) Yuma Fukushima, Takanori Haga, Wall-modeled LES of high Reynolds number channel flow using flux-reconstruction method, 34th Computational Fluid Dynamics Symposium, A03-3, 2020.

2) Takanori Haga, Taro Shimizu, Large-eddy simulations of a nitrogen jet under supercritical pressure by a flux-reconstruction method, 34th Computational Fluid Dynamics Symposium, B06-2, 2020.

3) Takanori Haga, Yuma Fukushima, Kazuma Tago, Toward Wall-modeled LES Using High-order Flux-reconstruction Method, Fluid Dynamics Conference / Aerospace Numerical Simulaiton Symposium 2020 Online, 2C01 (JSASS-2020-2081-A), 2020.

4) Kazuma Tago, Takanori Haga, Yuma Fukushima, Tsutsumi Seiji, Ryoji Takaki, Speed-up of high-order compressible flow on unstructured grids by GPU, Fluid Dynamics Conference / Aerospace Numerical Simulaiton Symposium 2020 Online, 2C05 (JSASS-2020-2085-A), 2020.

- Oral Presentations

1) Takanori Haga, Ryotaro Sakai, Yuma Fukushima, Mitsuhiro Murayama, Takashi Amemiya, Hiroyuki Ito, Unsteady Flow Analysis for NASA-CRM at Low-speed and High Angle-of-attack Conditions Using Flux-reconstruction Method, Fluid Dynamics Conference / Aerospace Numerical Simulaiton Symposium 2020 Online, 1A14, 2020.

Usage of JSS

Computational Information

• Process Parallelization Methods: MPI
• Thread Parallelization Methods: OpenMP
• Number of Processes: 32 - 11520
• Elapsed Time per Case: 27 Hour(s)

Resources Used(JSS2)

Fraction of Usage in Total Resources*1(%): 4.06

Details

Please refer to System Configuration of JSS2 for the system configuration and major specifications of JSS2.

*1: Fraction of Usage in Total Resources: Weighted average of three resource types (Computing, File System, and Archiver).

*2: Fraction of Usage：Percentage of usage relative to each resource used in one year.

Resources Used(JSS3)

Fraction of Usage in Total Resources*1(%): 3.87

Details

Please refer to System Configuration of JSS3 for the system configuration and major specifications of JSS3.

*1: Fraction of Usage in Total Resources: Weighted average of three resource types (Computing, File System, and Archiver).

*2: Fraction of Usage：Percentage of usage relative to each resource used in one year.

JAXA Supercomputer System Annual Report April 2020-March 2021