JAXA Supercomputer System Annual Report February 2021-January 2022

Report Number: R21ER0201

Subject Category: Space Technology

PDF available here

• Responsible Representative: Riko Oki, Director, Space technology Directorate I, Earth Observation Research Center
• Contact Information: Space technology Directorate I, Earth Observation Research Center, Takuji Kubota(kubota.takuji@jaxa.jp)
• Members: Koji Terasaki, Yingwen Chen, Kaya Kanemaru, Shunji Kotsuki, Hisashi Yashiro

Abstract

This study explores an effective use of satellite data including GSMaP and GPM/DPR through an advanced ensemble data assimilation method for improving numerical weather prediction (NWP) and pioneering a new precipitation product based on an NWP model and satellite observations, named as NICAM-LETKF JAXA Research Analysis (NEXRA).

Reference URL

Please refer to ‘JAXA Realtime weather watch‘.

Reasons and benefits of using JAXA Supercomputer System

In this study, the JSS3 is used for the NICAM-LETKF experiments to assimilate satellite observations and to conduct NWP model forecasts. The JSS2 is a necessary infrastructure for our study to conduct massive computations for the ensemble-based data assimilation and ensemble atmospheric simulations.

Achievements of the Year

We implemented a local particle filter with the LETKF, which is the data assimilation method used in the operational NEXRA system. The results showed generally improved analysis in the sparsely observed regions. This work is now under review in the EGU journal Geoscientific Model Development. We also implemented the hybrid background error covariance methods with the LETKF. In the LETKF, localization is usually applied to the observation error covariance matrix, but this study applied it to ensemble perturbations. The results showed that the optimal localization scale in the hybrid LETKF is longer than that in the conventional LETKF. We published a paper in the AMS journal Monthly Weather Review.

The high-resolution weather information data released: In FY2021, the weather information provided on the web site has been renewed. The 5-day forecast with the horizontal resolution at 112 km was provided til July, 2021, and the 5-day forecast with the horizontal resolution at 14 km started from July. The 5-day forecast weather information with initial time 00/06/12/18 UTC is continually provided on the web site. This high resolution forecast system is also able to provide the ensemble forecast when the severe weather systems, such as typhoons and heave rain events, are approaching to Japan region.

The ensemble data for research purpose: Currently, 5 ensemble forecasts with initial time at every 00 UTC are conducted continually. This ensemble forecast system started its operation since September 2021 and the ensemble forecast data are provided to the Today’s Earth project for developing the river run-off prediction system.

Fig.1: The accumulated precipitation comparison in between (a) NEXRA 112 km forecast, and (c) NETX 14 km forecast for the heavy rain event occurred in 2021.07.01-03. The initial time for the forecast is 2021.06.30 00 UTC. As shown in this figure, the precipitation pattern and accumulated precipitation are improved when the resolution becomes finer.

Fig.2: The time series of precipitation rate [mm/hr] around the Kanto region (34-35N; 138-140E) durning 2021.06.30-07.04 UTC. The initial time for simulations is 2021.06.30 00 UTC. The averaged region is where the heavy rain occurred (34-35N; 138-140E). Yellow line, red line, blue line, and gray lines denotes the hourly precipitation obtained from GSMaP, NEXRA 14 km routine forecast, mean of the ensemble forecast, and the ensemble forecast for each member. As shown in this figure, the 14 km routine NEXRA forecast show weak precipitation rate through the heavy rain event. In contrast, the large spread in the precipitation rate appears among the ensemble forecasts and one member captured the rain peak occurred in July 1st well. How these ensemble data can be used to provide the risk information is one important issue.

Fig.3: Spatial patterns of background RMSE for T (K) at the fourth model level (～500 hPa) for (a) LETKF and (b) LPFGM, averaged over February-April with RAOB and 1100-km localization scale. Panel (c) shows the difference between LETKF and LPFGM. Warm (cold) color indicates that the LPFGM is better (worse) than the LETKF. Adopted from Kotsuki et al. (2022).

Fig.4: First-guess (FG; i.e., 6-h forecast) RMSEs from 20-member experiments for temperature (K) at 500 hPa averaged over the 10-month period from March to December of 1986 as a function of the hybrid coefficient. The dashed line and solid lines indicate the RMSEs for the LETKF and hybrid LETKF, respectively, for c=20 (green), c=100 (blue), and c=365 (red). Adopted from Figure 7 of Kotsuki and Bishop (2022).

Publications

– Peer-reviewed papers

Kotsuki, S., and Bishop, H. C. (2022): Implementing Hybrid Background Error Covariance into the LETKF with Attenuation-based Localization: Experiments with a Simplified AGCM. Mon. Wea. Rev., 150, 283-302. doi: 10.1175/MWR-D-21-0174.1

Miyoshi, T., K. Terasaki, S. Kotsuki, S. Otsuka, Y- W- Chen, K. Kanemaru, K. Okamoto, K. Kondo, G- Y- Lien, H. Yashiro, H. Tomita, M. Sato, and E. Kalnay, 2022: Enhancing data assimilation of GPM observations. In: Silas M. (Eds) Precipitation Science, Measurement Remote Sensing, Microphysics, and Modeling. Elsevier, 787-804. doi:10.1016/B978-0-12-822973-6.00020-2

Terasaki, K. and T. Miyoshi, 2022: Ensemble Kalman Filter Experiments at 112-km and 28-km Resolution for the Record-Breaking Rainfall Event in Japan in July 2018. In: Park S.K., Xu L. (eds) Data Assimilation for Atmospheric, Oceanic and Hydrologic Applications (Vol. IV). Springer, Cham. 525-542. doi:10.1007/978-3-030-77722-7_20

Terasaki, K., and T. Miyoshi, 2022: A 1024-Member NICAM-LETKF Experiment for the July 2020 Heavy Rainfall Event. 18A, 6-9(TBA), doi:10.2151/sola.18A-002

– Invited Presentations

Takemasa Miyoshi, Weather Predictability and Data Assimilation: Perspectives Toward General Theory of Prediction, OIST-RIKEN Collaboration 1st Symposium: Green and blue planet -How can ecological research shape our future?, Okinawa, 2021/4/6

Takemasa Miyoshi, Seminar “Fusing Big Data and Big Computation in Numerical Weather Prediction”, Fluid Mechanics Unit (Professor Pinaki Chakraborty) Seminar, Online, 2021/4/20

Takemasa Miyoshi, Big Data, Big Computation, and Machine Learning in Numerical Weather Prediction, 14th International Conference on Mesoscale Convective Systems and High-Impact Weather in East Asia (ICMCS-XIV), Online, 2021/4/30

Kotsuki, S., Terasaki, K., Satoh, M., and Miyoshi, T.: Ensemble-Based Data Assimilation of GPM DPR Reflectivity into the Nonhydrostatic Icosahedral Atmospheric Model NICAM. JpGU 2021

Miyoshi Takemasa, Fusing Big Data and Big Computation in Numerical Weather Prediction, Data-Centric Engineering Webinar Series, Online, 2021/11/3

Kotsuki, S., and Bishop, H. C.: Implementing Hybrid Background Error Covariance into the LETKF with Attenuation-based Localization: Experiments with a Simplified AGCM. Data Assimilation Workshop

Takemasa Miyoshi, Enhancing Precipitation Prediction Algorithm by Data Assimilation of GPM Observations, PMM session of the joint PI meeting of JAXA Earth Observation Missions FY2021, 2022/1/12

– Oral Presentations

Takemasa Miyoshi, Takumi Honda, Arata Amemiya, Shigenori Otsuka, Yasumitsu Maejima, James Taylor, Hirofumi Tomita, Seiya Nishizawa, Kenta Sueki, Tsuyoshi Yamaura, Yutaka Ishikawa, Shinsuke Satoh, Tomoo Ushio, Kana Koike, Erika Hoshi, and Kengo Nakajima, Big Data Assimilation: Real-time Demonstration Experiment of 30-second-update Forecasting in Tokyo in August 2020, EGU2021, Online, 2021/4/26

Kotsuki, S., Oishi, K., and Miyoshi, T.: Improving the stability of the Local Particle Filter and Its Gaussian Mixture Extension: Experiments with an Intermediate AGCM. JpGU 2021

S. Ohishi, T. Hihara, H. Aiki, J. Ishizaka, Y. Miyazawa, M. Kachi, and T. Miyoshi, Improving the salinity structure by AOEI in an EnKF-based ocean data assimilation system, Japan Geoscience Union Meeting 2021, Online, June 2021

Takemasa Miyoshi, Improving Precipitation Prediction by Data Assimilation of GPM and Other Satellite Observations, Japan Geoscience Union Meeting, Online, 2021/6/3

Takemasa Miyoshi, Big Data Assimilation: Real-time Demonstration Experiment of 30-second-update Forecasting in Tokyo in August 2020, Japan Geoscience Union Meeting, Online, 2021/6/4

Takemasa Miyoshi, Experimental platform for design and advance evaluation of frequent satellite observations to innovate weather, ocean and land surface, Japan Geoscience Union Meeting, Online, 2021/6/4

Takemasa Miyoshi, Observing Systems Simulation Experiment (OSSE) Evaluating New Observing Systems Before Deployment, RCCS-cafe, Online, 2021/6/7

Takemasa Miyoshi, Big Data Assimilation: Real-time Demonstration Experiment of 30-second-update Forecasting in Tokyo in August 2020, Asia Oceania Geosciences Society virtual 2021 18th Annual meeting, Online, 2021/8/4

Takemasa Miyoshi, Improving Precipitation Prediction by Data Assimilation of Gpm and Other Satellite Observations, Asia Oceania Geosciences Society virtual 2021 18th Annual meeting, Online, 2021/8/4

Kotsuki, S., Oishi, K., and Miyoshi, T.: Improving the stability of the Local Particle Filter and Its Gaussian Mixture Extension: Experiments with an Intermediate AGCM. WCRP-WWRP Symposium on Data Assimilation and Reanalysis

Takemasa Miyoshi, Takumi Honda, Arata Amemiya, Shigenori Otsuka, Yasumitsu Maejima, James Taylor, Hirofumi Tomita, Seiya Nishizawa, Kenta Sueki, Tsuyoshi Yamaura, Yutaka Ishikawa, Shinsuke Satoh, Tomoo Ushio, Kana Koike, Erika Hoshi, and Kengo Nakajima, Real-time Demonstration Experiments of 30-second-update Forecasting in Tokyo in 2020 and 2021, WMO Joint WCRP-WWRP Symposium on Data Assimilation and Reanalysis, Online, 2021/9/17

– Poster Presentations

J. Taylor, A. Okazaki, T. Honda, S. Kotsuki, M. Yamaji, and T. Miyoshi Oversampling Reflectivity Observations from a Geostationary Precipitation Radar Satellite: Impact on Typhoon Forecasts within a Perfect Model OSSE Framework, AOGS, Online, August 2021

Kotsuki, S., Terasaki, K., Satoh, M., and Miyoshi, T.: Ensemble-Based Data Assimilation of GPM DPR Reflectivity into the Nonhydrostatic Icosahedral Atmospheric Model NICAM. WCRP-WWRP Symposium on Data Assimilation and Reanalysis

Terasaki, K., and T. Miyoshi, Including the spatial observation error correlation in data assimilation of AMSU-A radiances, WMO Joint WCRP-WWRP Symposium on Data Assimilation and Reanalysis, Online, 2021/9/17

Usage of JSS

Computational Information

• Process Parallelization Methods: MPI
• Thread Parallelization Methods: OpenMP
• Number of Processes: 1 – 2
• Elapsed Time per Case: 12 Hour(s)

JSS3 Resources Used

Fraction of Usage in Total Resources*1(%): 2.36

Details

Please refer to System Configuration of JSS3 for the system configuration and major specifications of JSS3.

*1: Fraction of Usage in Total Resources: Weighted average of three resource types (Computing, File System, and Archiver).

*2: Fraction of Usage：Percentage of usage relative to each resource used in one year.

ISV Software Licenses Used

*2: Fraction of Usage：Percentage of usage relative to each resource used in one year.

JAXA Supercomputer System Annual Report February 2021-January 2022