Large-scale computing challenge on 3D data visualization
JAXA Supercomputer System Annual Report April 2020-March 2021
Report Number: R20EFHC0301
Subject Category: Common Business
- Responsible Representative: Naoyuki Fujita, Manager, JAXA, Security and Information Systems Department, Supercomputer Division
- Contact Information: Yusuke Takeda(ytakeda@sci.hokudai.ac.jp)
- Members: Yusuke Takeda, Yasuhiro Iba, Takeshi Hoshino, Tomohiro Usui, Ryota Fukai
Abstract
High-resolution and large-size data obtained with our 3D imaging technique require extra-large computational resources, which makes their analysis and visualization impossible. With the aid of JSS3 large-scale challenge, we analyzed these large-scale (i.e., gigapixel- and teravoxel-scale) obtained data for the first time.
Reference URL
N/A
Reasons and benefits of using JAXA Supercomputer System
A scientific visualization application Paraview installed in JSS3 is the best solution for large-scale data visualization because it enables parallel remote rendering with little difficulty. Besides, our preliminary study suggests that the large-scale rendering with Paraview requires large memory size and GPU computing. Therefore, both large-capacity RAMs and high-performance GPUs of TOKI-RURI are the best resources for our large-scale visualization challenge.
Achievements of the Year
Rendering experiments on relatively small data (0.5 teravoxel) were initially performed to determine the most suitable resources of TOKI-RURI for large-scale visualization. As a result, rendering with LM or XM nodes failed with partial data loss. In contrast, rendering with ST nodes was successful without any rendering failure.
Following these results, 1.2 teravoxels data rendering was successfully performed in a pipeline with 100 ST nodes for the first time. Moreover, 4.5 gigapixel files were also successfully processed and visualized in the same pipeline. These technical findings are helpful for future sample observation and analysis in much super-higher resolution.
This challenge demonstrates that JSS3’s up-to-date large computational resources provide incredible solutions for visualizing such super-high resolution 3D image data. The use of JSS3 in this challenge was not a traditional one such as numerical computation and simulation. Therefore, our findings and achievements are also valuable for potential supercomputer needs.
Publications
N/A
Usage of JSS
Computational Information
- Process Parallelization Methods: MPI
- Thread Parallelization Methods: N/A
- Number of Processes: 50 – 100
- Elapsed Time per Case: 6 Hour(s)
Resources Used(JSS2)
Fraction of Usage in Total Resources*1(%): 0.00
Details
Please refer to System Configuration of JSS2 for the system configuration and major specifications of JSS2.
System Name | Amount of Core Time(core x hours) | Fraction of Usage*2(%) |
---|---|---|
SORA-MA | 0.00 | 0.00 |
SORA-PP | 0.00 | 0.00 |
SORA-LM | 0.00 | 0.00 |
SORA-TPP | 0.00 | 0.00 |
File System Name | Storage Assigned(GiB) | Fraction of Usage*2(%) |
---|---|---|
/home | 21.68 | 0.02 |
/data | 135.68 | 0.00 |
/ltmp | 2,083.33 | 0.18 |
Archiver Name | Storage Used(TiB) | Fraction of Usage*2(%) |
---|---|---|
J-SPACE | 0.00 | 0.00 |
*1: Fraction of Usage in Total Resources: Weighted average of three resource types (Computing, File System, and Archiver).
*2: Fraction of Usage:Percentage of usage relative to each resource used in one year.
Resources Used(JSS3)
Fraction of Usage in Total Resources*1(%): 0.48
Details
Please refer to System Configuration of JSS3 for the system configuration and major specifications of JSS3.
System Name | Amount of Core Time(core x hours) | Fraction of Usage*2(%) |
---|---|---|
TOKI-SORA | 0.00 | 0.00 |
TOKI-RURI | 530,357.59 | 3.04 |
TOKI-TRURI | 0.00 | 0.00 |
File System Name | Storage Assigned(GiB) | Fraction of Usage*2(%) |
---|---|---|
/home | 140.91 | 0.10 |
/data | 97,807.48 | 1.64 |
/ssd | 500.68 | 0.26 |
Archiver Name | Storage Used(TiB) | Fraction of Usage*2(%) |
---|---|---|
J-SPACE | 0.00 | 0.00 |
*1: Fraction of Usage in Total Resources: Weighted average of three resource types (Computing, File System, and Archiver).
*2: Fraction of Usage:Percentage of usage relative to each resource used in one year.
JAXA Supercomputer System Annual Report April 2020-March 2021