JAXA Supercomputer System Annual Report April 2019-March 2020

Report Number: R19EG3212

Subject Category: Research and Development

PDF available here

• Responsible Representative: Eiji Shima, Director, Research and Development Directorate, Research Unit III
• Contact Information: Takanori Haga, Research and Development Directorate, Research Unit III(haga.takanori@jaxa.jp)
• Members: Takayuki Ito, Ryoji Takaki, Seiji Tsutsumi, Hiroyuki Ito, Taro Shimizu, Junya Aono, Takanori Haga, Yuhi Morii, Masaharu Abe, Masayuki Kakehi, Mikiroh Motoe, Manabu Hishida, Hideyo Negishi, Yoichi Ohnishi, Miki Nishimoto, Yu Daimon, Ashvin Hosangadi, Takeru Fukuchi, Masaaki Ino, Osamu Fukasawa, Shinji Ohno, Andrea Zambon, Takenori Nakajima, Takashi Amemiya, Yutaka Umemura, Hironori Fujiwara, Hiroumi Tani, Keiichiro Fujimoto, Tetsufumi Ohmaru, Taroh Fukuda, Masashi Toyama, Kei Nishimura, Daiki Muto, Nozomu Kanno, Takeshi Watanabe

Abstract

To capture the unsteady phenomena in the liquid rocket engine, combustion large eddy simulations (LES) are carried out, and this evaluation tool is validated by comparing with a sub-scale test.

Reference URL

Please refer to ‘Research on software, computational engineering, and verification technology | JAXA | Research and Development Directorate‘.

Reasons and benefits of using JAXA Supercomputer System

Since the flow and combustion in rocket chambers are in a turbulent state and have nonstationary characteristics, LES analysis is essential. Even in this verification target, analysis calculation of about several million steps is required for grid of tens of millions of cells, so it is impossible to achieve the target without using supercomputer.

Achievements of the Year

1. Towards full-scale combustion chamber simulations, large eddy simulation (LES) of an oxygen/hydrogen subscale combustion chamber is performed. JAXA’s in-house CFD solver LS-FLOW is used in the simulation. A new wall model for reacting flows is applied to characterize the near-wall flow, reducing about 1000 times of computational cost compared with the conventional wall resolved LES. For combustion modeling, the non-adiabatic flamelet progress variable approach is used. The number of grid point is approximately 83 million. Figure 1 shows the instantaneous temperature field. The present simulation captures the complex turbulent combustion field and flame/wall interaction structures.

2. In order to realize accurate and efficient combustion LES, we have developed a high-order unstructured grid solver LS-FLOW-HO based on the flux reconstruction method. Localized Laplacian artificial diffusivity (LLAV) is introduced to stably capture the discontinuous interface of the multi-component, thermally perfect gas while maintaining the turbulence resolution. The Flamelet model, which can handle many chemical species efficiently, is adopted as the combustion model. Figure 2 shows the computational results of the methane / oxygen single injector combustor. The number of grid cells is 110,496, and the degrees of freedom are 2,983,392 (p2, third-order) and 7,071,744 (p3, fourth-order). Even with a coarse grid of about 3 cells in the shear layer, the development of turbulence has been captured with the fourth-order scheme. The parallelization method is MPI / OpenMP hybrid. It uses FX100’s 2048 cores (144 processes times 8 threads) and the computation time is about 72 hours. In the future, tuning for further acceleration will be performed, and GPU will be supported by OpenACC.

Fig.1: Instantaneous temperature field of Oxygen / hydrogen subscale combustion chamber.

Fig.2: Close up view of the temperature distribution near the injector (recess 12 mm). (Top: p3, fourth order, bottom: p2, third order)

Publications

– Peer-reviewed papers

Muto, D., Daimon, Y., Shimizu, T., Negishi, H., An equilibrium wall model for reacting turbulent flows with heat transfer, International Journal of Heat and Mass Transfer, 141, 2019, 1187-1195, 10.1016/j.ijheatmasstransfer.2019.05.101.

– Invited Presentations

Haga, T., Shimizu, T., Toward high-fidelity large-eddy simulation of liquid rocket combustors using flux-reconstruction method, Japan-Korea CFD workshop 2019.

– Oral Presentations

1) Haga, T., Muto, D., Daimon, Y., Negishi, H., Shimizu, T., Haidn, O., Combustion modeling study for GCH4/LOX and GCH4/GOX single element combustion chambers, Sonderforschungsbereich/Transregio 40 – Annual Report, 2019.

2) Muto, D., Daimon, Y., Shimizu, T., Negishi, H., Wall-modeled large eddy simulation for predicting wall heat flux in a rocket combustion chamber, 8th EUCASS, 2019, 10.13009/EUCASS2019-252.

Usage of JSS2

Computational Information

• Process Parallelization Methods: MPI
• Thread Parallelization Methods: OpenMP
• Number of Processes: 144 – 3840
• Elapsed Time per Case: 480 Hour(s)

Resources Used

Fraction of Usage in Total Resources*1(%): 2.26

Details

Please refer to System Configuration of JSS2 for the system configuration and major specifications of JSS2.

*1: Fraction of Usage in Total Resources: Weighted average of three resource types (Computing, File System, and Archiver).

*2: Fraction of Usage：Percentage of usage relative to each resource used in one year.

JAXA Supercomputer System Annual Report April 2019-March 2020