Examination of ISAS’s scientific satellitedata processing using JSS
JAXA Supercomputer System Annual Report February 2022-January 2023
Report Number: R22EEU30800
Subject Category: Space and Astronautical Science
- Responsible Representative: NAKAHIRA, Satoshi, Center for Science-satellite Operation and Data Archive, ISAS
- Contact Information: NAKAHIRA, Satoshi(nakahira.satoshi@jaxa.jp)
- Members: Ken Ebisawa, Satoshi Nakahira
Abstract
Examine and demonstrate technologies for parallel processing of scientific satellite data at ISAS.
Reference URL
N/A
Reasons and benefits of using JAXA Supercomputer System
The resources required for fast data reprocessing are increasing due to the sophistication of observation equipment and longer operations.
Achievements of the Year
Digital altitude data (~60k files, approx. 2 TB) of the lunar surface acquired by KAGUYA up to around 2009 were processed for conversion into HEALpix for global data used in the field of astronomy. Simple parallelization and the use of sparse matrices in TensorFlow enabled fast conversion of the all-moon spherical data into an array of approximately 700 GB of data.
Publications
– Peer-reviewed papers
“Development of HEALPix-based KAGUYA’s lunar desital elevation database and data provision system”, KOBAYASHI, Nene; KAJIURA, Rio; NAKAHIRA, Satoshi; YAMAMOTO, Yukio; UCHIDA, H., Akihito; EBISAWA, Ken; Journal of Space Science Informatics Japan: (In review)
Usage of JSS
Computational Information
- Process Parallelization Methods: Since the input data is divided, job submission optimization tools were used to execute the jobs simultaneously.
- Thread Parallelization Methods: N/A
- Number of Processes: 100 – 500
- Elapsed Time per Case: 6 Hour(s)
JSS3 Resources Used
Fraction of Usage in Total Resources*1(%): 0.00
Details
Please refer to System Configuration of JSS3 for the system configuration and major specifications of JSS3.
System Name | CPU Resources Used(Core x Hours) | Fraction of Usage*2(%) |
---|---|---|
TOKI-SORA | 20.73 | 0.00 |
TOKI-ST | 29510.05 | 0.03 |
TOKI-GP | 0.00 | 0.00 |
TOKI-XM | 0.00 | 0.00 |
TOKI-LM | 0.00 | 0.00 |
TOKI-TST | 0.00 | 0.00 |
TOKI-TGP | 0.00 | 0.00 |
TOKI-TLM | 0.00 | 0.00 |
File System Name | Storage Assigned(GiB) | Fraction of Usage*2(%) |
---|---|---|
/home | 28.33 | 0.03 |
/data and /data2 | 17920.00 | 0.14 |
/ssd | 801.33 | 0.11 |
Archiver Name | Storage Used(TiB) | Fraction of Usage*2(%) |
---|---|---|
J-SPACE | 5.94 | 0.03 |
*1: Fraction of Usage in Total Resources: Weighted average of three resource types (Computing, File System, and Archiver).
*2: Fraction of Usage:Percentage of usage relative to each resource used in one year.
ISV Software Licenses Used
ISV Software Licenses Used(Hours) | Fraction of Usage*2(%) | |
---|---|---|
ISV Software Licenses(Total) | 0.00 | 0.00 |
*2: Fraction of Usage:Percentage of usage relative to each resource used in one year.
JAXA Supercomputer System Annual Report February 2022-January 2023